Papers with Representation Learning
Probabilistic FastText for Multi-Sense Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Probabilistic FastText model for word embeddings captures word senses, sub-word structure, and uncertainty information. |
| Approach: | They propose a model for word embeddings that captures multiple word senses . they represent each word with a Gaussian mixture density, with each vector representing an n-gram . |
| Outcome: | The proposed model outperforms dictionary-level probabilistic embeddings on word-similarity benchmarks. |
Cross-lingual Semantic Representation for NLP with UCCA (2020.coling-tutorials)
Copied to clipboard
| Challenge: | introductory tutorial to UCCA, a symbolic meaning representation for semantic representations. |
| Approach: | This tutorial introduces UCCA, a cross-linguistically applicable framework for semantic representation . it will provide a detailed introduction to the UCca annotation guidelines, design philosophy and available resources . |
| Outcome: | The tutorial will provide a detailed introduction to the UCCA framework and compare it to other meaning representations. |
Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine (D19-62)
Copied to clipboard
| Challenge: | Existing work has trained medical embeddings to rep-resent medical concepts using specific medical data. |
| Approach: | They use intrinsic methods to evaluate pre-trained word embeddings from the various fields of medicine as defined by their ICD-9 systems. |
| Outcome: | The results show that the embeddings perform better in one field of medicine than in other fields. |
More Discriminative Sentence Embeddings via Semantic Graph Smoothing (2024.eacl-short)
Copied to clipboard
| Challenge: | Text categorization is a natural language processing task that involves arranging texts into coherent groups based on their content. |
| Approach: | They propose to use semantic graph smoothing to enhance sentence embeddings from pretrained models to improve results for supervised and unsupervised document categorization tasks. |
| Outcome: | The proposed method improves sentences embeddings for supervised and unsupervised document categorization tasks. |
When Specialization Helps: Using Pooled Contextualized Embeddings to Detect Chemical and Biomedical Entities in Spanish (D19-57)
Copied to clipboard
| Challenge: | Existing work on pharmacological entities requires manual annotation of these units. |
| Approach: | They propose an approach to task 1 of the PharmaCoNER Challenge to recognize pharmacological entities on a spanish corpus. |
| Outcome: | The proposed approach achieves 89.76% score on a spanish corpus based on pre-trained embeddings and 90.52% score on domain-specific embeddables. |
Automatic Learning of Modality Exclusivity Norms with Crosslingual Word Embeddings (2020.starsem-1)
Copied to clipboard
| Challenge: | Normative studies on modality for English words are relatively common . however, they are limited to a relatively small number of languages and require costly ratings. |
| Approach: | They aim to learn a mapping between word embeddings and modality norms by training on a high-resource language and testing on . monolingual and crosslingual word embeds are used to predict modality association scores . |
| Outcome: | The proposed model predicts modality associations even when trained on an English resource and tested on a completely unseen language. |
ABLE: Agency-BeLiefs Embedding to Address Stereotypical Bias through Awareness Instead of Obliviousness (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent studies in Natural Language Processing (NLP) have unveiled a concerning issue: stereotypical biases associated with demographic groups are prevalent. |
| Approach: | They propose an approach that actively encodes stereotypical biases into the embedding space by integrating stereotypes into a model that acquires agency and belief scores rather than directly representing stereotypes. |
| Outcome: | The proposed model can learn agency and belief stereotypes while preserving the language model’s proficiency. |
Towards Layered Events and Schema Representations in Long Documents (2021.naacl-srw)
Copied to clipboard
| Challenge: | a thesis aims to explore the use of event extraction in literary texts . event extraction is a challenging domain based on its variety of genres . |
| Approach: | They propose to use event extraction to extract semantic information from literary texts . they propose to build on sequences of event embeddings to form schema embeddables . |
| Outcome: | The proposed approach will allow comparisons between sections of documents and entire literary works. |
Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we discuss the cutting-edge research results and existing challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
| Approach: | This tutorial presents cutting-edge research results and current challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
| Outcome: | This paper reviews the cutting-edge research results and current challenges related to spatial language understanding including semantic annotations, existing corpora, symbolic and sub-symbolic representations, qualitative spatial reasoning, spatial common sense, deep and structured learning models. |
BioReddit: Word Embeddings for User-Generated Biomedical NLP (D19-62)
Copied to clipboard
| Challenge: | a corpus of medical-themed posts was scrapped from Reddit to train word embeddings on downstream tasks. |
| Approach: | They propose to train word embeddings from a corpus of medical forums from reddit scrapping posts from medical-themed subreddits. |
| Outcome: | The proposed system outperforms embeddings trained on general purpose data or on scientific papers when applied on user-generated content. |
A Deeper Look into Dependency-Based Word Embeddings (N18-4)
Copied to clipboard
| Challenge: | Word embeddings trained with dependency contexts excel at different tasks, and enhanced dependencies often improve performance. |
| Approach: | They propose to use dependency-based word embeddings to capture semantic similarity rather than relatedness. |
| Outcome: | The results show that word embeddings trained with Universal and Stanford dependencies excel at different tasks and that enhanced dependencies often improve performance. |
Improving Hate Speech Detection by Fusing Textual and User Interaction Representations in Online Communities (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing studies on toxic content in online communities are limited by the scarcity of data that align textual content with comprehensive social interactions. |
| Approach: | They propose a user-aware hate speech detection framework that effectively fuses textual semantics with social interaction representations to provide pragmatic context for disambiguation. |
| Outcome: | The proposed framework outperforms strong text-only baselines by over 3.6%, validating the critical role of social context in enhancing detection accuracy. |
Spatial and Temporal Language Understanding: Representation, Reasoning, and Grounding (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
| Approach: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
| Outcome: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
Counterfactuals to Control Latent Disentangled Text Representations for Style Transfer (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for unsupervised text style transfer focus on transferring a specific attribute, but this technique has never been explored in natural language generation tasks. |
| Approach: | They propose a counterfactual-based method to modify latent representations by posing a ‘what-if’ scenario. |
| Outcome: | The proposed method is tested on multiple attribute transfer tasks like Sentiment, Formality and Excitement to support the hypothesis. |
Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation (D18-1)
Copied to clipboard
Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, Benjamin Van Durme
| Challenge: | a plethora of new natural language inference datasets has been created in recent years . however, these datasets do not provide clear insight into what type of reasoning or inference a model may be performing. |
| Approach: | They propose to recast 13 existing natural language inference datasets into a common structure. |
| Outcome: | The proposed datasets provide insight into how well a sentence representation captures distinct types of reasoning. |
Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings (D19-1)
Copied to clipboard
| Challenge: | Word embeddings are increasingly used for automatic detection of semantic change, but a robust evaluation and systematic comparison of the choices involved has been lacking. |
| Approach: | They propose a new evaluation framework for semantic change detection using whole time series and a Twitter dataset spanning 5.5 years. |
| Outcome: | The proposed framework shows that using whole time series is preferable over continuously trained embeddings for long time periods and that the reference point matters. |
A Partially Rule-Based Approach to AMR Generation (N19-3)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a representation of a sentence as a labeled graph . because of these abstractions, it can be difficult to generate from AMR back to a fluent English sentence . |
| Approach: | They propose a new approach to generating English text from Abstract Meaning Representation (AMR) it is largely rule-based, supplemented by a language model and simple statistical linearization models . they also address difficulties of automatically evaluating AMR generation systems . |
| Outcome: | The proposed approach produces a fluent English sentence with a high quality . it is difficult to generate from an AMR back to a sentence which preserves original meaning . |
ConShift: Sense-based Language Variation Analysis using Flexible Alignment (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for semantic variation analysis are limited due to the limited evaluation datasets available for word-level and sense-level variants. |
| Approach: | They propose a family of alignment-based algorithms that enable semantic variation analysis at the sense-level. |
| Outcome: | The proposed algorithms can detect multiple sense-level language variations while providing explanations through visualization of related concepts. |
SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposes (2023.acl-demo)
Copied to clipboard
| Challenge: | SanskritShala is a neural-based Sanskrit NLP toolkit that is available as a web-based application . |
| Approach: | They propose a neural Sanskrit NLP toolkit that facilitates linguistic analyses for word segmentation, morphological tagging, dependency parsing, and compound type identification. |
| Outcome: | The proposed toolkit reports state-of-the-art performance on benchmark datasets . it is built with easy-to-use interactive data annotation features . |
Natural Language Generation: Recently Learned Lessons, Directions for Semantic Representation-based Approaches, and the Case of Brazilian Portuguese Language (P19-2)
Copied to clipboard
| Challenge: | Natural Language Generation (NLG) is a promising area in Natural Language Processing (NLP) . |
| Approach: | They present a review of the literature on Natural Language Generation in Brazilian Portuguese. |
| Outcome: | The proposed approaches are based on the Abstract Meaning Representation formalism and have potential future directions. |
Contrasting distinct structured views to learn sentence embeddings (2021.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods to build sentence embeddings rely on a similar Recurrent Neural Network (RNN) heterogeneity of performances across models and tasks makes us assume some structures might be better adapted given the considered task or sentence. |
| Approach: | They propose a self-supervised method that builds sentence embeddings from syntactic structures . they hypothesize that some linguistic representations might be better adapted given the task . |
| Outcome: | The proposed method outperforms comparable methods on several tasks from standard sentence embedding benchmarks. |
Contextualized Word Representations from Distant Supervision with and for NER (D19-55)
Copied to clipboard
| Challenge: | Existing word embeddings for named entity recognition are stacked with traditional ones for downstream tasks. |
| Approach: | They propose a special type of contextualized word representation that is learned from distant supervision annotations and dedicated to named entity recognition. |
| Outcome: | The proposed representation surpasses the existing representations and is complementary to existing embeddings. |
Computational Discovery of Chiasmus in Ancient Religious Text (2025.naacl-short)
Copied to clipboard
| Challenge: | chiasmus, or chiastic units, is a debated literary device in biblical texts . a computational approach to detect chiastes is shown to be efficient, but not efficient . |
| Approach: | They propose a computational approach to detect chiasmus within Biblical passages . they leverage neural embeddings to capture lexical and semantic patterns associated with chiastics - using annotators to review a subset of the detected patterns. |
| Outcome: | The proposed method achieves high inter-annotator agreement and system accuracy of 0.80 at verse level and 0.60 at half-verse level. |
Morphology-Aware Meta-Embeddings for Tamil (2021.naacl-srw)
Copied to clipboard
| Challenge: | In this work, we focus on producing morphologically enhanced word embeddings for Tamil, a highly agglutinative South Indian language with rich morphology that remains low-resource with regards to NLP tasks. |
| Approach: | They present a first-ever word analogy dataset for Tamil using a rules-based segmenter and meta-embedding techniques. |
| Outcome: | The proposed embeddings outperform baselines on the word analogy task by 16% and appear to mitigate a trade-off between semantic and morphological accuracy. |
DadmaTools: Natural Language Processing Toolkit for Persian Language (2022.naacl-demo)
Copied to clipboard
| Challenge: | Existing tools for Persian language processing are based on conventional non-neural models and do not take full advantage of the latest developments. |
| Approach: | They propose to use a Python neural pipeline for Persian text processing tasks . they use 'parsBERT' to fine-tune the Python pipeline using the PerDT dataset . |
| Outcome: | The proposed toolkit can achieve state-of-the-art performance on multiple NLP tasks. |
Improving Embedding-based Large-scale Retrieval via Label Enhancement (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for large-scale retrieval are trained with 0-1 hard labels that indicate whether a query is relevant to a document, ignoring rich information of the relevance degree. |
| Approach: | They propose to introduce label enhancement for the first time to characterize query-document relevance degree by embedding label distribution into contextual embeddables. |
| Outcome: | The proposed method significantly outperforms existing retrieval models and its counterparts equipped with two alternative methods on English and Chinese large-scale retrieval tasks. |
Exploiting WordNet Synset and Hypernym Representations for Answer Selection (2020.aacl-main)
Copied to clipboard
| Challenge: | Answer selection (AS) is a challenging subtask of document-based question answering (DQA). |
| Approach: | They propose to use WordNet to enrich the word representation and sentence encoding to incorporate similarity scores of two concepts that share synset or hypernym relations into the attention mechanism. |
| Outcome: | The proposed model outperforms existing state-of-the-art models on the public WikiQA and SelQA datasets and significantly improves the baseline system. |
Metric for Automatic Machine Translation Evaluation based on Universal Sentence Representations (N18-4)
Copied to clipboard
| Challenge: | Sentence representations can capture information that cannot be captured by local features based on character or word Ngrams. |
| Approach: | They propose a supervised regression model using universal sentence representations capable of capturing information that cannot be captured by local features based on character or word Ngrams. |
| Outcome: | The proposed model achieves state-of-the-art performance with only sentence representation features . |
On Sentence Representations for Propaganda Detection: From Handcrafted Features to Word Embeddings (D19-50)
Copied to clipboard
| Challenge: | The rise of fake and hyperpartisan news on social media and online news outlets calls for improved automatic detection of propaganda in texts. |
| Approach: | They propose to use handcrafted features and learn dense semantic representations to detect propaganda in sentence-level and with random undersampling of the majority class (non-propaganda) |
| Outcome: | The proposed system achieves a ranking of 10 among 25 participants, with 59.5 F1-score. |
Decoding Brain Activity Associated with Literal and Metaphoric Sentence Comprehension Using Distributional Semantic Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing research has focused on applying semantic models to decode brain activity associated with the meaning of individual words. |
| Approach: | They evaluate a range of semantic models to capture metaphor processing in the brain . they found that compositional models and word embeddings capture differences in the processing of literal and metaphoric sentences . |
| Outcome: | The proposed models capture differences in the processing of literal and metaphoric sentences, providing support for the idea that the literal meaning is not fully accessible during familiar metaphor comprehension. |
Representing ELMo embeddings as two-dimensional text online (2021.eacl-demos)
Copied to clipboard
| Challenge: | ELMoViz module adds support for contextualized embedding architectures, in particular for token embeddable word models. |
| Approach: | They propose to add a module to the free and open-source WebVectors toolkit which provides lexical hyperlinks to word representations in static embedding models. |
| Outcome: | The ELMoViz module adds support for contextualized embedding architectures, in particular for ELMa models. |
SNAP-BATNET: Cascading Author Profiling and Social Network Graphs for Suicide Ideation Detection on Social Media (N19-3)
Copied to clipboard
Rohan Mishra, Pradyumn Prakhar Sinha, Ramit Sawhney, Debanjan Mahata, Puneet Mathur, Rajiv Ratn Shah
| Challenge: | Suicide is a leading cause of death among youth worldwide and currently only uses text-based cues to detect suicidal ideation. |
| Approach: | They propose a deep learning based model to extract text-based features from tweets and a novel Feature Stacking approach to combine other community-based information. |
| Outcome: | The proposed model outperforms existing models on an annotated dataset of tweets using a three-phase strategy and proposes a novel Feature Stacking approach to combine other community-based information such as historical author profiling and graph embeddings. |
Domain-Specific Word Embeddings with Structure Prediction (2023.tacl-1)
Copied to clipboard
| Challenge: | Current word embedding methods do not provide a way to use or predict information on structure between sub-corpora, time or domain. |
| Approach: | They propose a word embedding method that provides general word representations for the whole corpus, domain-specific representations and embeddable alignment simultaneously. |
| Outcome: | The proposed method provides better performance than baselines on a dataset of science and philosophy articles. |
From Text to Lexicon: Bridging the Gap between Word Embeddings and Lexical Resources (C18-1)
Copied to clipboard
| Challenge: | Distributional word representations are omnipresent in modern NLP. |
| Approach: | They propose to combine lemmatization and part of speech (POS) typing to improve word embedding performance. |
| Outcome: | The proposed methods improve word embedding performance on verbs and verbs. |
Learning Lexical Subspaces in a Distributional Vector Space (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing word embeddings that can cluster distributionally related words are weak, but they can be used to cluster words that might not be semantically similar. |
| Approach: | They propose a framework that injects lexical-semantic relations into distributional word embeddings by defining subspaces of the distributional vector space in which a lexically related relation should hold. |
| Outcome: | The proposed framework outperforms existing systems on relatedness and hypernymy tasks while being competitive on word similarity tasks. |
Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Cross-domain sentiment analysis is a hot topic in research and industry . domain-invariant representation learning (DIRL) is used to learn a feature representation across domains . but, when label distribution P(Y) shifts across domain, it degrades performance . |
| Approach: | They propose a domain-invariant representation learning framework to improve cross-domain sentiment analysis performance. |
| Outcome: | The proposed model is easy to transfer existing models to the proposed model. |
Towards Incremental Learning of Word Embeddings Using Context Informativeness (P19-2)
Copied to clipboard
| Challenge: | In this paper, we investigate the task of learning word embeddings from very sparse data in an incremental, cognitively-plausible way. |
| Approach: | They propose a model that incorporates informativeness into a proposed model of nonce learning, using it for context selection and learning rate modulation. |
| Outcome: | The proposed model is based on a proposed model of nonce learning, and it performs well on the task of learning new words from definitions and potentially uninformative contexts. |
LINSPECTOR WEB: A Multilingual Probing Suite for Word Representations (D19-3)
Copied to clipboard
| Challenge: | LINSPECTOR WEB is an open source multilingual inspector to analyze word embeddings. |
| Approach: | They propose to use LINSPECTOR WEB to analyze word embeddings in 28 languages. |
| Outcome: | The system performs 16 simple linguistic probing tasks for a diverse set of 28 languages. |
Identifying Emergent Research Trends by Key Authors and Phrases (C18-1)
Copied to clipboard
| Challenge: | Existing methods to identify emergent research trends have been employed to generate corpora of large corporata. |
| Approach: | They propose an embedded trend detection framework which integrates hypothesis that important phrases are written by important authors within a field and vice versa. |
| Outcome: | The proposed framework outperforms baselines based on text centrality or citations over two large datasets of scientific articles. |
Does My Representation Capture X? Probe-Ably (2021.acl-demo)
Copied to clipboard
| Challenge: | Probing (or diagnostic classification) has become a popular strategy for investigating whether a given set of intermediate features is present in the representations of neural models. |
| Approach: | They propose to use an extendable probing framework to automate the application of probing methods to the user’s inputs. |
| Outcome: | The proposed framework automates the application of probing methods to the user’s inputs. |
Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semantics (2022.aacl-short)
Copied to clipboard
| Challenge: | Current neural models for Chinese story generation struggle to generate high-quality long text narratives due to ambiguity in syntactically parsing the Chinese language. |
| Approach: | They propose a framework that enhances the feature capturing mechanism by informing the generation model of dependencies between words and additionally augmenting the semantic representation learning through synonym denoising training. |
| Outcome: | The proposed framework outperforms the state-of-the-art Chinese generation models on all evaluation metrics, showing that it enhances dependency and semantic representation learning. |
CLER: Cross-task Learning with Expert Representation to Generalize Reading and Understanding (D19-58)
Copied to clipboard
| Challenge: | In-domain datasets are used to train and validate our model, and other out-of-domain data are used for validation. |
| Approach: | They propose a model which uses cross-task learning with expert representation for the generalization of reading and understanding. |
| Outcome: | The proposed model achieved an average F1 score of 66.1 % in the out-of-domain setting, which is a 4.3 percentage point improvement over the official BERT baseline model. |
„Mann“ is to “Donna” as「国王」is to « Reine » Adapting the Analogy Task for Multilingual and Contextual Embeddings (2023.starsem-1)
Copied to clipboard
| Challenge: | a lack of comparable multilingual benchmarks and a consensual evaluation protocol for contextual models remains an open question. |
| Approach: | They propose a multilingual analogy dataset and evaluate human and contextual embedding performance. |
| Outcome: | The proposed dataset evaluates human and contextual embedding models on the analogy task. |
Greenback Bears and Fiscal Hawks: Finance is a Jungle and Text Embeddings Must Adapt (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Financial documents are filled with specialized terminology, arcane jargon, and curious acronyms that pose challenges for general-purpose text embeddings. |
| Approach: | They propose to fine tune financial text embeddings finetuned on a carefully constructed dataset of 14.3M query-passage pairs including both public and proprietary financial documents. |
| Outcome: | The proposed embeddings achieve Recall@1 of 62.8% on a held-out test set, vs. only 39.2% for the best general-purpose text embeddING from OpenAI. |
Overcoming Poor Word Embeddings with Word Definitions (2021.starsem-1)
Copied to clipboard
| Challenge: | Modern natural language understanding models depend on pretrained word embeddings, but applications may need to reason about words that were never or rarely seen during pretraining. |
| Approach: | They propose a method to improve a model's ability to learn to use definitions in natural text to overcome this handicap. |
| Outcome: | The proposed model learns to use definitions in natural text to overcome this handicap. |
Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae (P19-3)
Copied to clipboard
| Challenge: | Embeddings are a fundamental component of many modern machine learning and natural language processing models. |
| Approach: | They propose a tool for visualizing embedding spaces using parametric projections . they demonstrate the power of Parallax and propose % task-oriented approach . |
| Outcome: | The proposed tool is based on two-dimensional projections without interpretable semantics . it enhances interpretability and allows for more fine-grained analysis . |
Denoising Word Embeddings by Averaging in a Shared Space (2021.starsem-1)
Copied to clipboard
| Challenge: | Continuous word embeddings have been introduced several years ago as a standard building block for NLP tasks. |
| Approach: | They propose a method of fusing word embeddings that were trained on the same corpus but with different initializations. |
| Outcome: | The proposed method improves word embeddings on a range of tasks. |
MINER: Multi-Interest Matching Network for News Recommendation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods learn a single user embedding from user’s historical behaviors to represent the reading interest. |
| Approach: | They propose a poly attention scheme to learn multiple interest vectors for each user, which encodes the different aspects of user interest. |
| Outcome: | The proposed approach significantly outperforms existing state-of-the-art methods on the MIND news recommendation benchmark. |
Dense Node Representation for Geolocation (D19-55)
Copied to clipboard
| Challenge: | Existing methods for geolocation use sparse adjacency matrices of connections, which grow exponentially with the number of users. |
| Approach: | They propose two methods to learn continuous node representations from social media posts and textual user mentions. |
| Outcome: | The proposed methods improve performance over previous sparse graph representations. |
Polarized-VAE: Proximity Based Disentangled Representation Learning for Text Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of real-world data focus on attribute labels or unsupervised methods that manipulate factorization in the latent space of models such as the variational autoencoder (VAE). |
| Approach: | They propose an approach that disentangles select attributes in the latent space based on proximity measures reflecting the similarity between data points with respect to these attributes. |
| Outcome: | The proposed method outperforms the VAE baseline and is competitive with state-of-the-art approaches while being more a general framework applicable to other attribute disentanglement tasks. |
Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Information (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing approaches to mitigate social biases require explicit annotation of demographic information for each sample. |
| Approach: | They propose a method that leverages predefined demographic texts and incorporates a regularization term during the fine-tuning process to mitigate bias in language models. |
| Outcome: | The proposed method outperforms debiasing methods with limited demographic-annotated data. |
How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models excel at capturing semantic correlations within utterance embeddings but fail to determine specific causal relationships. |
| Approach: | They propose to incorporate i.i.d. noise terms into conversation process to build a structural causal model . they propose to use unstructured conversation data to facilitate deep learning . |
| Outcome: | The proposed approach can be implemented in unstructured conversation data and a synthetic dataset that includes i.i.d. noise. |
Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing tree-based models require handannotated data to be trained. |
| Approach: | They propose a tree-based model that learns its composition function together with its structure. |
| Outcome: | The proposed model outperforms existing models on downstream tasks and is competitive with Bert base model. |
Mittens: an Extension of GloVe for Learning Domain-Specialized Representations (N18-2)
Copied to clipboard
| Challenge: | We show that the resulting representations can lead to faster learning and better results on a variety of tasks. |
| Approach: | They propose a simple extension of the GloVe representation learning model that starts with general-purpose representations and updates them based on specialized data sets. |
| Outcome: | The proposed model synthesizes general-purpose representations with specialized data while remaining faithful to the original space. |
Embedding-based Scientific Literature Discovery in a Text Editor Application (2020.acl-demos)
Copied to clipboard
| Challenge: | Despite the availability of powerful search engines and text editing software, discovering relevant papers and integrating the knowledge into a manuscript remain complex tasks associated with high cognitive load. |
| Approach: | They propose to combine text editing and literature discovery in an interactive user interface with a search engine that couples Boolean keyword filtering with nearest neighbor search over text embeddings. |
| Outcome: | The proposed application combines text editing and literature discovery in an interactive user interface. |
Automatic Derivation of Semantic Representations for Thai Serial Verb Constructions: A Grammar-Based Approach (2024.acl-srw)
Copied to clipboard
| Challenge: | Using rich semantic representations for Thai Serial Verb Constructions (SVCs) is time-consuming and manual annotation is preferred. |
| Approach: | They propose to implement an HPSG analysis for Thai Serial Verb Constructions (SVCs) they use a DELPH-IN computational grammar to generate appropriate representations from syntactic features. |
| Outcome: | The proposed grammar increases verified coverage of Thai SVCs by 73% and decreases ambiguity by 46% on held-out data. |
Personalized Filled-pause Generation with Group-wise Prediction Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Disfluency generation is a method to generate personalized filled pauses (FPs) compared with fluent text generation, it is difficult to predict them because of the sparsity of position and frequency difference between more and less frequently used FPs. |
| Approach: | They propose a method to generate personalized filled pauses (FPs) by group-wise prediction models. |
| Outcome: | The proposed method generates personalized filled pauses (FPs) with group-wise prediction models. |
Embedding Strategies for Specialized Domains: Application to Clinical Entity Recognition (P19-2)
Copied to clipboard
| Challenge: | Off-the-shelf word embeddings tend to perform poorly on texts from specialized domains such as clinical reports. |
| Approach: | They combine off-the-shelf contextual embeddings with static word2vec embedders trained on a small in-domain corpus built from task data to reach and sometimes outperform representations learned from a large corpus in the medical domain. |
| Outcome: | The proposed embedding strategies outperform representations learned from a large corpus in the medical domain. |
Capturing the Relationship Between Sentence Triplets for LLM and Human-Generated Texts to Enhance Sentence Embeddings (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in building sentence embedding models have centered on replacing traditional human-generated text datasets with those generated by LLMs. |
| Approach: | They propose a loss function that incorporates Positive-Negative sample Augmentation within the contrastive learning objective to enhance sentence embeddings using both human and LLM-generated datasets. |
| Outcome: | The proposed model mitigates the sentence anisotropy problem in Wikipedia corpus and improves Spearman’s correlation in standard Semantic Textual Similarity (STS) tasks (+1.47% compared to CLHAIF). |
Can Network Embedding of Distributional Thesaurus Be Combined with Word Vectors for Better Representation? (N18-1)
Copied to clipboard
| Challenge: | Distributed representations of words learned from text have proved to be successful in various natural language processing tasks. |
| Approach: | They propose to embed a distributional thesaurus network into dense word vectors and compare them to state-of-the-art word representations. |
| Outcome: | The proposed representations improve performance against state-of-the-art word representations even without handcrafted lexical resources. |
Building Location Embeddings from Physical Trajectories and Textual Representations (2020.aacl-main)
Copied to clipboard
| Challenge: | Using a dataset consisting of the location trajectories of 729 students over a seven month period, we investigate whether embeddings can represent aspects such as location presence or location functionality. |
| Approach: | They propose to use location embeddings to generate embeddables of sequences of locations a student has visited to identify surface properties captured in the representations. |
| Outcome: | The proposed models can be used to predict depression levels and area of study, and can be applied to complex tasks such as predicting area of studies and depression levels. |
Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources (N18-1)
Copied to clipboard
| Challenge: | Word vector specialisation is a portable, light-weight approach to fine-tuning distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet. |
| Approach: | They propose a constraint-driven vector space specialisation method that embeds external knowledge into lexical resources into a deep neural network to specialise unseen words. |
| Outcome: | The proposed method preserves useful linguistic knowledge for seen words while propagating external signal to unseen words to improve their vector representations. |
Representation Learning for Resource-Constrained Keyphrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art keyphrase generation methods depend on large annotated datasets, limiting their performance in domains with limited annotation data. |
| Approach: | They propose a method that first identifies salient information using retrieval-based corpus-level statistics and then learns a task-specific intermediate representation based on a pre-trained language model. |
| Outcome: | The proposed method improves keyphrase generation and zero-shot domain adaptation on multiple keyphrase benchmarks. |
OoMMix: Out-of-manifold Regularization in Contextual Embedding Space for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies on neural networks with pre-trained weights focus on low-dimensional subspace where the embedding vectors computed from input words are located. |
| Approach: | They propose an approach to find and regularize the remainder of the space, referred to as out-of-manifold, which cannot be accessed through the words. |
| Outcome: | The proposed approach is able to fine-tune the out-of-manifold embedding space on text classification benchmarks. |
Vector Space Interpolation for Query Expansion (2022.aacl-short)
Copied to clipboard
| Challenge: | Topic-sensitive query set expansion is crucial for queries related to sensitive and emerging topics. |
| Approach: | They propose a method for topic-sensitive query set expansion using vector space interpolation. |
| Outcome: | The proposed method generates new queries about the sensitive topic by incorporating set diversity, which is not captured by traditional sentence-level augmentation methods such as paraphrasing or back-translation. |
Acoustic Individual Identification of White-Faced Capuchin Monkeys Using Joint Multi-Species Embeddings (2025.acl-short)
Copied to clipboard
| Challenge: | acoustic identification of animals is an essential task for conservation and wildlife monitoring . but, many methods for automatic identification are hindered by lack of data . |
| Approach: | They explore cross-species pre-training to address the task of individual classification in white-faced capuchin monkeys. |
| Outcome: | The proposed methods can be used to identify calls from individual monkeys using acoustic embeddings from birds and humans. |
Contextualized context2vec (D19-55)
Copied to clipboard
| Challenge: | Lexical substitution ranks substitution candidates from the viewpoint of paraphrasability for a target word in a given sentence. |
| Approach: | They propose a method that combines two approaches to contextualize word embeddings for lexical substitution. |
| Outcome: | The proposed method outperforms the current state-of-the-art method and assigns English proficiency levels to all target words and substitution candidates. |
ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that traditional retrievers struggle with reasoningintensive tasks such as personal assistants and scientific research. |
| Approach: | They propose a new data synthesis method that overcomes the triviality problem prevalent in previous synthetic datasets and propose 'ReMixer', a data fusion method that generates 82K high-quality training samples. |
| Outcome: | The proposed model outperforms existing models on reasoning-intensive retrieval tasks. |
Leveraging distributed representations and lexico-syntactic fixedness for token-level prediction of the idiomaticity of English verb-noun combinations (P18-2)
Copied to clipboard
| Challenge: | Verb-noun combinations (VNCs) are ambiguous between literal and idiomatic usages in English. |
| Approach: | They propose and evaluate models for classifying verb-noun combinations as idiomatic or literal, based on averaging word embeddings and a variety of approaches to forming distributed representations. |
| Outcome: | The proposed model outperforms a previous model based on skip-thoughts and averaging word embeddings. |
RevUp: Revise and Update Information Bottleneck for Event Representation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing external (“side”) semantic knowledge has been shown to result in more expressive computational event models. |
| Approach: | They propose a semi-supervised information bottleneck-based discrete latent variable model that reparameterizes discrete variables with auxiliary continuous latent variables and a light-weight hierarchical structure. |
| Outcome: | The proposed model outperforms existing models on multiple datasets. |
Generalizing Word Embeddings using Bag of Subwords (D18-1)
Copied to clipboard
| Challenge: | Existing word embeddings techniques have a fixed vocabulary, i.e., they can only provide vectors over a finite set of common words that appear frequently in a given corpus. |
| Approach: | They propose a subword-level word vector generation model that views words as bags of character n-grams and provides good vectors for rare or unseen words. |
| Outcome: | The proposed model performs state-of-the-art in English word similarity task and in joint prediction of part-of speech tag and morphosyntactic attributes in 23 languages. |
Parameter-free Sentence Embedding via Orthogonal Basis (D19-1)
Copied to clipboard
| Challenge: | Existing methods to build sentence embeddings are parameterized and require training to optimize their parameters. |
| Approach: | They propose a non-parameterized method to combine pre-trained word embeddings into sentence representations using an orthogonal basis of the word vector subspace and its surrounding context. |
| Outcome: | The proposed method shows superior performance on 11 downstream NLP tasks and is competitive to other methods relying on large amounts of labelled data or prolonged training time. |
Querying Word Embeddings for Similarity and Relatedness (N18-1)
Copied to clipboard
| Challenge: | Word2Vec embeddings have become popular representations of word meaning . similarity between two words is often assumed to be a direction-less measure, whereas relatedness is inherently directional. |
| Approach: | They propose to use word embeddings to predict asymmetric association between words from a dataset of production norms to generate thematically related words. |
| Outcome: | The proposed model predicts asymmetric association between words from a recently published dataset of production norms. |
Gender Bias in Contextualized Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that training word embeddings in large corpora could lead to encoding societal biases present in these human-produced data. |
| Approach: | They conduct several intrinsic analyses to quantify, analyze and mitigate gender bias exhibited in ELMo’s contextualized word vectors. |
| Outcome: | The proposed method mitigates gender bias on WinoBias probing corpus and demonstrates that it can be implemented in other systems. |
Learning Universal Authorship Representations (2021.emnlp-main)
Copied to clipboard
Rafael A. Rivera-Soto, Olivia Elizabeth Miano, Juanita Ordonez, Barry Y. Chen, Aleem Khan, Marcus Bishop, Nicholas Andrews
| Challenge: | authorship verification has traditionally relied on modeling stylometric linguistic properties . but neural methods introduce a tradeoff: they obviate the need for manual feature design . |
| Approach: | They propose to use domain-specific features to improve authorship representations . they propose to study Amazon reviews, fanfiction short stories, and Reddit comments . |
| Outcome: | The proposed methods outperform existing methods in large-scale authorship verification scenarios. |
Explaining Word Embeddings via Disentangled Representation (2020.aacl-main)
Copied to clipboard
| Challenge: | Disentangled representations are known to represent interpretable factors in separated dimensions. |
| Approach: | They propose to transform dense word vectors into disentangled embeddings with improved interpretability by encoding polysemous semantics separately. |
| Outcome: | The proposed model can be encoded into multiple sub-embeddings or sub-areas and generates more efficient and effective features for natural language processing. |
Protein-STORY: Semantic Text-Oriented Representation Yields biologically meaningful Protein embeddings (2026.acl-short)
Copied to clipboard
| Challenge: | Unsupervised representation learning relying on sequence data often overlooks decades of expert-curated biological knowledge stored in textual formats. |
| Approach: | They propose a pipeline that synthesizes protein embeddings from diverse, multi-source text descriptions and a network architecture that integrates high-fidelity functional and structural insights into a unified representation. |
| Outcome: | The proposed pipeline outperforms existing models on diverse downstream tasks (+2 pts F1) and enables zero-shot text-prompted protein search. |
Cross-Lingual Learning-to-Rank with Shared Representations (N18-2)
Copied to clipboard
| Challenge: | Cross-lingual information retrieval (CLIR) is a document retrieval task where the documents are written in a language different from that of the user's query. |
| Approach: | They propose a large-scale dataset derived from Wikipedia to support CLIR research in 25 languages. |
| Outcome: | The proposed model can improve the results of Swahili-English CLIR in Japanese and Japanese. |
What if This Modified That? Syntactic Interventions with Counterfactual Embeddings (2021.findings-acl)
Copied to clipboard
| Challenge: | Prior art aims to uncover meaningful properties within model representations, but it is unclear how faithfully such probes portray information that the models actually use. |
| Approach: | They propose a technique for generating counterfactual embeddings within models . they produce evidence that some models use a tree-distancelike representation of syntax . |
| Outcome: | The proposed technique produces evidence that some models use tree-distancelike representations of syntax in downstream prediction tasks. |
Monitoring geometrical properties of word embeddings for detecting the emergence of new topics. (2021.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that slow emerging topics are often detected too late . a positive correlation is linked with event-like topics while a negative correlation is a sign of emergence. |
| Approach: | They propose to monitor words representation in embedding space and use one of its geometrical properties to characterize the emergence of topics. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two public datasets of press and scientific articles. |
Supervised and Nonlinear Alignment of Two Embedding Spaces for Dictionary Induction in Low Resourced Languages (D19-1)
Copied to clipboard
| Challenge: | Existing methods for mapping monolingual word embeddings into another are based on anchor points and unsupervised methods are more adversarial. |
| Approach: | They propose a noise-tolerant piecewise linear technique to learn a non-linear mapping between two monolingual word embedding vector spaces. |
| Outcome: | The proposed method outperforms the state-of-the-art in lower resourced settings with an average of 3.7% improvement of precision @10 across 14 mostly low resourced languages. |
Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to information retrieval (IR) and natural language processing (NLP) use contextual language models, which can improve both synonymy and polysemy problems associated with words. |
| Approach: | They propose an ultra-high dimensional representation scheme equipped with directly controllable sparsity and a bucketing method where embeddings from multiple layers of BERT are selected/merged to represent diverse linguistic aspects. |
| Outcome: | The proposed representation scheme outperforms sparse models with MS MARCO and TREC CAR, and shows that it is highly efficient for storage and search. |
Automated essay scoring with string kernels and word embeddings (P18-2)
Copied to clipboard
| Challenge: | Existing approaches to automatic essay scoring use low-level character n-gram features. |
| Approach: | They propose to combine string kernels and word embeddings for automatic essay scoring. |
| Outcome: | The proposed method outperforms state-of-the-art deep learning methods in Arabic dialect identification and native language identification tasks. |
A Frame-based Sentence Representation for Machine Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing machine learning approaches do not have above semantic knowledge to address complicated MRC questions. |
| Approach: | They propose a frame-based Sentence Representation method which integrates frame semantic knowledge to facilitate sentence modelling. |
| Outcome: | The proposed method performs better than state-of-the-art methods on machine reading comprehension task. |
Crowdsourcing Question-Answer Meaning Representations (N18-2)
Copied to clipboard
| Challenge: | Existing datasets for predicate-argument relationships are lacking highly skilled and trained annotators. |
| Approach: | They propose a crowdsourcing scheme to generate question-answer pairs that represent predicate-argument relationships in sentences as a set of question-announcer pairs. |
| Outcome: | The proposed model covers the vast majority of predicate-argument relationships in existing datasets along with many previously under-resourced ones, including implicit arguments and relations. |
Addressing Noise in Multidialectal Word Embeddings (P18-2)
Copied to clipboard
| Challenge: | Dialectal Arabic (DA) is problematically noisy and lacks a large corpus of non-noisy words. |
| Approach: | They propose to use word embedding tools to maximize the informative content leveraged in each training sentence and analyze methods for representing disparate dialects in one embeddable space. |
| Outcome: | The proposed methods improve performance on low and high frequency words while preserving accuracy on low frequency forms. |
Deep Generative Model for Joint Alignment and Word Representation (N18-1)
Copied to clipboard
| Challenge: | EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. |
| Approach: | They exploit translation as a distributional context and embed words as posterior probability densities, rather than point estimates, which allows them to compare words in context using a measure of overlap between distributions. |
| Outcome: | The proposed model performs on a range of lexical semantics tasks and achieves competitive results on benchmarks including natural language inference, paraphrasing, and text similarity. |
Event Pattern-Instance Graph: A Multi-Round Role Representation Learning Strategy for Document-Level Event Argument Extraction (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing role-based span selection strategies ignore interrelations between events . authors propose a multi-round role representation learning strategy for document-level event argument extraction . |
| Approach: | They propose a pattern-instance graph to capture role semantics embedded in various associations . they also propose re-inventing the role representations learned from previous analyzed documents . |
| Outcome: | The proposed model captures role semantics embedded in various associations . iteratively updates representations of role nodes and edges to enrich their semantic information . the model improves prediction performance in subsequent rounds of span selection . |
A Systematic Study of Leveraging Subword Information for Learning Word Representations (N19-1)
Copied to clipboard
| Challenge: | Existing word representation models for morphologically rich languages use subword-level information, but their systematic comparative analysis across typologically diverse languages and tasks is still missing. |
| Approach: | They propose a framework for learning subword-informed word representations that allows for easy experimentation with different segmentation and composition components. |
| Outcome: | The proposed framework allows for easy experimentation with different segmentation and composition components, as well as advanced techniques based on position embeddings and self-attention. |
Learning to Answer Psychological Questionnaire for Personality Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text-based personality detection research relies on data-driven approaches to implicitly capture personality cues in online posts lacking the guidance of psychological knowledge. |
| Approach: | They propose a model to capture key information in texts and a questionnaire to help the user to make a personality assessment. |
| Outcome: | The proposed model captures key information in texts and a questionnaire and can be used to improve personality prediction. |
Align Voting Behavior with Public Statements for Legislator Representation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies rely on roll call data to estimate political preference of legislators. |
| Approach: | They propose to integrate voting behavior and public statements on Twitter to jointly model legislators. |
| Outcome: | The proposed model improves on the task of roll call vote prediction . it also shows that the model captures nuances in statements . |
Measuring Idiomaticity in Text Embedding Models with epsilon-compositionality (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies on compositionality of text embedding models have limited understanding of the principle . idioms have traditionally been seen as non-compositional . |
| Approach: | They propose to use formal definitions to define compositionality in text embedding models . they find that most models differentiate between idiomatic and non-idiomatic phrases . |
| Outcome: | The proposed model is able to differentiate between idiomatic and non-idiomatic phrases, the authors show . |
Extracting Commonsense Properties from Embeddings with Limited Human Guidance (P18-2)
Copied to clipboard
| Challenge: | Existing methods for learning common sense from text require dozens of hand-annotated frames to connect the property to how it is indirectly reflected in text. |
| Approach: | They propose a method for extracting object-property comparisons from pre-trained embeddings. |
| Outcome: | The proposed approach exceeds previous work but requires less hand-annotated knowledge. |
Disentangling Meaning and Language Components in Diverse Multilingual Sentence Embeddings (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing studies have reported language specificity in multilingual sentence embeddings, resulting in language-specific subspaces. |
| Approach: | They propose to disentangle multilingual sentence embeddings into language-dependent and language-agnostic components to improve cross-lingual similarity estimation. |
| Outcome: | The proposed methods improve cross-lingual similarity estimation across multiple embeddings. |
Narrative Embedding: Re-Contextualization Through Attention (2021.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach to narrative event representation uses attention to re-contextualize events across the whole story . a recent study shows that attention is used to attach event semantics to tokens . |
| Approach: | They propose an unsupervised approach to narrative event representation using attention to re-contextualize events across the whole story. |
| Outcome: | The proposed approach achieves state of the art performance on multiple choice and story cloze tasks. |
Gaussian Mixture Latent Vector Grammars (P18-1)
Copied to clipboard
| Challenge: | Existing models of latent variable grammars are not observable in treebanks, so latent variables are learned using expectation-maximization. |
| Approach: | They propose a new framework that extends latent variable grammars such that each nonterminal symbol is associated with a continuous vector space representing the set of (infinitely many) subtypes of the nonterminals. |
| Outcome: | The proposed framework can achieve competitive accuracies in part-of-speech tagging and constituency parsing. |
A Retrofitting Model for Incorporating Semantic Relations into Word Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | Existing word embedding models mix semantic similarity with other types of relatedness. |
| Approach: | They propose a model that leverages relational knowledge available in a knowledge resource to improve word embeddings. |
| Outcome: | The proposed model improves word embeddings on synonymy, antonymy and hypernymy relations in WordNet and significantly improves lexical entailment detection task. |
Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing word embedding methods for natural language processing are limited in their ability to produce dense word embeds. |
| Approach: | They propose a word embedding SentiVec which is infused with sentiment information from a lexical resource and outperforms baselines on subjectivity-sensitive tasks. |
| Outcome: | The proposed word embedding SentiVec outperforms baselines on subjectivity-sensitive tasks. |
Diachronic word embeddings and semantic shifts: a survey (C18-1)
Copied to clipboard
| Challenge: | Existing methods for tracing time-related semantic shifts with word embedding models lack the cohesion, common terminology and shared practices of more established areas of natural language processing. |
| Approach: | They propose several axes along which these methods can be compared and propose a framework for comparison. |
| Outcome: | The proposed methods are compared with existing methods and outline their main challenges and potential applications. |
GEMNET: Effective Gated Gazetteer Representations for Recognizing Complex Entities in Low-context Input (2021.naacl-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is difficult in real-world settings due to short texts, emerging entities, and complex entities. |
| Approach: | They propose a flexible Gazetteer Representation encoder and a Mixture-of-Experts gating network for gazetteer knowledge integration. |
| Outcome: | The proposed approach shows large gains (up to +49% F1) in recognizing difficult entities compared to baselines. |
‘Lighter’ Can Still Be Dark: Modeling Comparative Color Descriptions (P18-2)
Copied to clipboard
| Challenge: | Multimodal approaches to object recognition ground adjectives and nouns from text using comparative adjectives. |
| Approach: | They propose a new paradigm of grounding comparative adjectives within the realm of color descriptions by using a vector model. |
| Outcome: | The proposed model generates representations of comparative adjectives with an average accuracy of 0.65 cosine similarity to the desired direction of change. |
Hierarchical Representation-based Dynamic Reasoning Network for Biomedical Question Answering (2022.coling-1)
Copied to clipboard
Jianguo Mao, Jiyuan Zhang, Zengfeng Zeng, Weihua Peng, Wenbin Jiang, Xiangdong Wang, Hong Liu, Yajuan Lyu
| Challenge: | Existing models of biomedical question answering are limited in their ability to predict answers . a new model improves the performance of existing models, but the code will be released after the paper is published. |
| Approach: | They propose a hierarchical representation-based dynamic reasoning network to solve biomedical problems. |
| Outcome: | The proposed model significantly improves on three mainstream biomedical datasets . the code will be released after the paper is published . |
Attentive Multiview Text Representation for Differential Diagnosis (2021.acl-short)
Copied to clipboard
| Challenge: | Using data from the Undiagnosed Diseases Network, we find that NLP algorithms can reproduce the performance of clinical experts in the task of differential diagnosis. |
| Approach: | They propose a text representation approach that can combine different views of the same input to improve ranking. |
| Outcome: | The proposed model outperforms several ranking approaches by effectively prioritizing and combining representations obtained from traditional and recent text representation techniques. |
VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models on Machine Reading Comprehension (MRC) require complex model architecture for effectively modeling long texts with paragraph representation and classification, making inference computationally inefficient for production use. |
| Approach: | They propose a novel Gaussian distribution-based paragraph representation for Machine Reading Comprehension (MRC) that is light-weight and parallel-efficient. |
| Outcome: | The proposed model can achieve comparable performance on Wikipedia-based (NQ) and TechNotes (TechQA) with a state-of-the-art (SOTA) complex document modeling approach while being 16 times faster, demonstrating the efficiency of the proposed model. |
Norm of Word Embedding Encodes Information Gain (2023.emnlp-main)
Copied to clipboard
| Challenge: | Distributed representations of words encode lexical semantic information, but what type of information is encoded and how? |
| Approach: | They propose to use the squared norm of static word embedding to encode the information gain conveyed by the word. |
| Outcome: | The proposed model can encode the information gain of a word in a language model or neural network. |
Learning Slice-Aware Representations with Mixture of Attentions (2021.findings-acl)
Copied to clipboard
| Challenge: | Real-world machine learning systems are achieving excellent performance in terms of coarse-grained metrics like overall accuracy and F-1 score. |
| Approach: | They extend slice-based learning (SBL) with a mixture of attentions to learn slice-aware dual attentive representations. |
| Outcome: | The proposed approach outperforms the baseline method and the original SBL approach on monitored slices with two natural language understanding tasks. |
Detecting Cybersecurity Events from Noisy Short Text (N19-1)
Copied to clipboard
Semih Yagcioglu, Mehmet Saygin Seyfioglu, Begum Citamak, Batuhan Bardak, Seren Guldamlasioglu, Azmi Yuksel, Emin Islam Tatli
| Challenge: | Using domain-specific word embeddings, we propose a method to detect cyber security events from noisy short text. |
| Approach: | They propose a method that leverages domain-specific word embeddings and task-specific features to detect cyber security events from tweets. |
| Outcome: | The proposed model outperforms both baselines and traditional models on a dataset of 2K tweets and manually annotates them. |
Dependency parsing with structure preserving embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | Modern neural approaches to dependency parsing are trained to predict a tree structure by learning a contextual representation for tokens in a sentence and a head–dependent scoring function. |
| Approach: | They propose to combine a contextual representation for tokens and a head–dependent scoring function to learn interpretable representations by training a parser to explicitly preserve structural properties of a tree. |
| Outcome: | The proposed approach yields strong tree distance preservation and parsing performance on par with a competitive graph-based parser. |
Measuring and Mitigating Racial Bias in Embedding Models: A Comparative Study for Law Enforcement Retrieval (2026.acl-industry)
Copied to clipboard
| Challenge: | Embedding models are often used for semantic retrieval in high-stakes domains such as law enforcement . racial descriptors affect similarity scores and retrieval rankings for semantically identical crime incidents . |
| Approach: | They propose to use racial descriptors to measure r&d bias in embedding models . they compute similarity scores between crime incidents and simple law enforcement queries . |
| Outcome: | The proposed methods show that racial descriptors affect similarity scores and retrieval rankings for semantically identical crime incidents. |
Documents Representation via Generalized Coupled Tensor Chain with the Rotation Group constraint (2021.findings-acl)
Copied to clipboard
| Challenge: | despite the diversity of linguistic structures, vector embedding models lack order-preserving properties . current methods for learning linguistic structure can be expensive and time-consuming . |
| Approach: | They propose a method for embedding documents and words in rotation group . they capture word order and higher-order word interactions . |
| Outcome: | The proposed model achieves the best results in document classification benchmarks. |
A Timestep aware Sentence Embedding and Acme Coverage for Brief but Informative Title Generation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for title generation are based on timestep aware sentence embeddings, but they are not effective for generating a title with appropriate information in the content. |
| Approach: | They propose a Timestep aware Sentence Embedding mechanism which refreshes the sentences’ embeddings with corresponding key words in different decoding timesteps. |
| Outcome: | The proposed framework outperforms existing methods on various title generation tasks and the evaluation scores are significantly higher than previous approaches. |
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work analyzes, quantifies, and mitigates language model biases such as gender, race or religion-related stereotypes in static word embeddings and contextual representations. |
| Approach: | They explain the complexity of gender and language around it and examine how current representations perpetuate harms associated with binary gender. |
| Outcome: | The proposed model and dataset biases perpetuate harms associated with the treatment of gender as binary in English language technologies. |
Embedding Words as Distributions with a Bayesian Skip-gram Model (C18-1)
Copied to clipboard
| Challenge: | Rather than assuming that word embeddings are fixed across the entire text collection, we generate them from word-specific prior densities for each word. |
| Approach: | They propose a method for embedding words as probability densities in a low-dimensional space from a word-specific prior density for each occurrence of a given word. |
| Outcome: | The proposed method can encode word as a distribution on a range of benchmarks and is comparable to Gaussian embeddings. |
Soft Representation Learning for Sparse Transfer (P19-1)
Copied to clipboard
| Challenge: | Using adversarial training, we can “soft-code” shared and private spaces to avoid sparse sharing. |
| Approach: | They propose to use adversarial training to “soft-code” shared and private spaces to avoid the shared space gets too sparse. |
| Outcome: | The proposed architecture avoids sparse sharing of shared and private spaces, and also deals with low-quality input. |
Vec2Sent: Probing Sentence Embeddings with Natural Language Generation (2020.coling-main)
Copied to clipboard
| Challenge: | a new unsupervised probing task is able to retrieve black-box sentence embeddings . a variety of problems surround probing tasks, including manual construing . |
| Approach: | They propose a method to generate black-box sentence embeddings by conditionally generating from them . they also illustrate how the language generated from different encoders differs . |
| Outcome: | The proposed probing task improves the performance of black-box sentence embeddings . the proposed task is based on a conditional natural language generation approach . |
Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization (P19-1)
Copied to clipboard
Paul Pu Liang, Zhun Liu, Yao-Hung Hubert Tsai, Qibin Zhao, Ruslan Salakhutdinov, Louis-Philippe Morency
| Challenge: | Existing methods to regularize multimodal data are imperfect due to imperfect modalities, missing entries or noise corruption. |
| Approach: | They propose a method to regularize multimodal data by tensor rank minimization . they use correlations between time and modalities to generate low-rank tenses . |
| Outcome: | The proposed model achieves good results across various levels of imperfection. |
No Word Embedding Model Is Perfect: Evaluating the Representation Accuracy for Social Bias in the Media (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has relied on word embedding bias measures, such as WEAT, but these methods can be inaccurate due to several representation issues, such low-resource settings and token frequency differences. |
| Approach: | They propose to use WEAT to quantify social bias in US online news articles and embed embedding algorithms to account for the aforementioned issues. |
| Outcome: | The proposed algorithms do not match the literature, but they reduce the gap. |
Construction of a Japanese Word Similarity Dataset (L18-1)
Copied to clipboard
| Challenge: | evaluating distributed word representations in languages that do not have such resources is difficult . et al., 2015: distributed word represent a sparse vector indicating the word itself or the context of the word. |
| Approach: | They constructed a Japanese word similarity dataset to evaluate distributed representations in Japanese. |
| Outcome: | a Japanese word similarity dataset is the first resource that can be used to evaluate distributed representations in Japanese . the dataset contains various parts of speech and includes rare words in addition to common words . |
Towards Lossless Encoding of Sentences (P19-1)
Copied to clipboard
| Challenge: | Existing methods for encoding text into lossless representations focus on performing well on downstream tasks and are unable to reconstruct original sequence from learned embedding. |
| Approach: | They propose a lossless method for encoding long sequences of texts into feature rich representations by recursive autoencoding. |
| Outcome: | The proposed method performs well on sentiment analysis and sentiment classification tasks. |
comp-syn: Perceptually Grounded Word Embeddings with Color (2020.coling-main)
Copied to clipboard
Bhargav Srinivasa Desikan, Tasker Hull, Ethan Nadler, Douglas Guilbeault, Aabir Abubakar Kar, Mark Chu, Donald Ruggiero Lo Sardo
| Challenge: | Existing approaches to natural language processing ignore embodied sensory aspects of language. |
| Approach: | They propose a Python package that provides word embeddings based on Google Image search results. |
| Outcome: | The proposed package provides word embeddings based on the color distributions of Google Image search results. |
Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on suspense have only sporadically been used in story generation systems. |
| Approach: | They propose a hierarchical language model that computes surprise and uncertainty reduction over story representations and annotated short stories. |
| Outcome: | The proposed model can predict suspense over story representations or probability distributions, and predicts suspensity in movie synopses. |
Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings (P18-1)
Copied to clipboard
| Challenge: | Neural machine translation uses source and target word embeddings to improve translation quality . source and targeted word embeds are at the two ends of a long information processing procedure . |
| Approach: | They propose a method to shorten the distance between source and target words in neural machine translation by bridging source and targeting word embeddings. |
| Outcome: | The proposed method shortens the distance between source and target words in neural machine translation and strengthens their association. |
Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies reveal a security threat to natural language processing models, called the Backdoor Attack. |
| Approach: | They propose to hack a model by modifying one single word embedding vector without sacrificing accuracy on clean samples. |
| Outcome: | The proposed method is more efficient and stealthier on sentiment analysis and sentence-pair classification tasks. |
Understanding Undesirable Word Embedding Associations (P19-1)
Copied to clipboard
| Challenge: | Word embeddings are often criticized for capturing undesirable word associations such as gender stereotypes. |
| Approach: | They propose to use subspace projection to debias vectors post hoc using a model that implicitly does matrix factorization to debunk gender bias. |
| Outcome: | The proposed test overestimates gender bias in word embeddings by using subspace projection, a method that is widely used in training. |
Card-660: Cambridge Rare Word Dataset - a Reliable Benchmark for Infrequent Word Representation Models (D18-1)
Copied to clipboard
| Challenge: | Existing benchmarks for rare word representation are lacking for evaluation and comparison . a task-based evaluation does not provide a solid basis for comparing different models . |
| Approach: | They propose to use an expert-annotated word similarity dataset to evaluate rare word representation techniques. |
| Outcome: | The proposed dataset provides a reliable benchmark for rare word representation techniques. |
Multilingual Factor Analysis (P19-1)
Copied to clipboard
| Challenge: | Existing methods for multilingual word embeddings are based on the observation that word embeds exhibit similar structures across languages. |
| Approach: | They propose a latent variable-based model that fits a multilingual dictionary to learn multilingual word representations offline. |
| Outcome: | The proposed model is robust to noise in the embedding space making it suitable for distributed representations learned from noisy corpora. |
An Evaluation of Disentangled Representation Learning for Texts (2021.findings-acl)
Copied to clipboard
| Challenge: | Disentangled representations of texts encode information pertaining to different aspects of the text in separate vector embeddings. |
| Approach: | They propose to use a highly-structured natural language dataset to evaluate disentangled representations for texts. |
| Outcome: | The proposed models are well-suited for learning disentangled representations of texts on a synthetic natural language dataset. |
AMR dependency parsing with a typed semantic algebra (P18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representations (AMRs) are graphs which describe the predicate-argument structure of a sentence. |
| Approach: | They propose a semantic parser which parses strings into tree representations of the compositional structure of an AMR graph. |
| Outcome: | The proposed parser outperforms baselines and standard neural techniques for supertagging and dependency tree parsing. |
Measuring Intersectional Biases in Historical Documents (2023.findings-acl)
Copied to clipboard
Nadav Borenstein, Karolina Stanczak, Thea Rolskov, Natacha Klein Käfer, Natália da Silva Perez, Isabelle Augenstein
| Challenge: | digitised historical documents suffer from errors introduced by optical character recognition (OCR) and are written in an archaic language. |
| Approach: | They investigate the continuities and transformations of bias in Caribbean historical newspapers during the colonial era . they use distributional semantics models and word embeddings to measure gender, race, and intersectional biases. |
| Outcome: | The authors show that gender and racial biases are interdependent and their intersection triggers distinct effects. |
Dynamic Meta-Embeddings for Improved Sentence Representations (D18-1)
Copied to clipboard
| Challenge: | A sprawling literature has emerged about what word embeddings are most useful for which tasks . word embed-ding is a technique that can be used to learn word-level meaning representations for a variety of tasks. |
| Approach: | They propose a method for supervised learning of embedding ensembles that leads to state-of-the-art performance on a variety of tasks. |
| Outcome: | The proposed method leads to state-of-the-art performance on a variety of tasks. |
Reinforced Product Metadata Selection for Helpfulness Assessment of Customer Reviews (D19-1)
Copied to clipboard
| Challenge: | a helpful review is largely concerned with the metadata of its target product . a selector learns from both the key-value product metadata and one of its reviews to take an action . |
| Approach: | They propose a framework that uses product metadata to assess helpfulness of free-text reviews . they use two real-world datasets from amazon.com and Yelp.com to test the framework . |
| Outcome: | The proposed framework can achieve state-of-the-art performance with substantial improvements . it uses two real-world datasets from Amazon.com and Yelp.com . |
Learning Invariant Representations of Social Media Users (D19-1)
Copied to clipboard
| Challenge: | Existing methods for learning to compare social media users fail to generalize to new users or even to previously known users. |
| Approach: | They propose a procedure to learn a mapping from short episodes of user activity to a vector space in which the distance between points captures the similarity of the corresponding users’ invariant features. |
| Outcome: | The proposed procedure can be applied to users not seen at training time and enables efficient comparisons of users in the resulting vector space. |
Preposition Sense Disambiguation and Representation (D18-1)
Copied to clipboard
| Challenge: | Prepositions are highly polysemous and their variegated senses encode significant semantic information. |
| Approach: | They match each preposition’s context and their interplay to the geometry of the word vectors to the left and right of the preposition. |
| Outcome: | The proposed algorithm is comparable to and better than state-of-the-art on two benchmark datasets. |
Distributed Representations of Emotion Categories in Emotion Space (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on emotion detection focus on how to improve performance of models . however, emotion relations are ignored in one-hot representations . |
| Approach: | They propose a framework to learn distributed representations for emotion categories in emotion space from a given emotion classification dataset. |
| Outcome: | The proposed representations can express emotion relations much better than word vectors in semantic space. |
Multi-Granularity Representations of Dialog (D19-1)
Copied to clipboard
| Challenge: | Neural models of dialog rely on generalized latent representations of language. |
| Approach: | They propose a training procedure which explicitly learns multiple representations of language at several levels of granularity. |
| Outcome: | The proposed training procedure significantly improves performance on the next utterance retrieval task using the MultiWOZ dataset and the Ubuntu dialog corpus. |
A Unified Feature Representation for Lexical Connotations (2021.eacl-main)
Copied to clipboard
| Challenge: | ideological attitudes and stance are often expressed through subtle meanings of words and phrases. |
| Approach: | They propose a method for lexical representations that capture connotations within the embedding space . they define six new fine-grained connotation aspects for nouns and adjectives . |
| Outcome: | The proposed method improves stance detection when data is limited. |
Learning Disentangled Textual Representations via Statistical Measures of Similarity (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to disentangle a sensitive attribute from textual representations require training and multiple parameter updates. |
| Approach: | They propose a family of regularizers for learning disentangled representations that do not require training. |
| Outcome: | The proposed regularizers are faster and faster and achieve better results when combined with pretrained and randomly initialized text encoders. |
Factors Influencing the Surprising Instability of Word Embeddings (N18-1)
Copied to clipboard
| Challenge: | Word embeddings are low-dimensional, dense vector representations that capture semantic properties of words. |
| Approach: | They examine the stability of word embeddings by examining their properties and analyzing their effects on downstream tasks. |
| Outcome: | The results show that even high frequency words exhibit substantial instability, which can have implications for downstream tasks. |
Generalizing Natural Language Analysis through Span-relation Representations (2020.acl-main)
Copied to clipboard
| Challenge: | a large number of natural language processing tasks are generated with specially designed architectures. |
| Approach: | They propose to represent a wide variety of tasks in a single unified format . they perform extensive experiments to demonstrate benefits of multi-task learning . |
| Outcome: | The proposed model performs comparable to state-of-the-art models on 10 tasks . it also shows that it can analyze differences and similarities in how the model handles different tasks compared to other models . |
PRAM: An End-to-end Prototype-based Representation Alignment Model for Zero-resource Cross-lingual Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to address the named entity recognition problem are limited and lack explicit optimization specific to the task. |
| Approach: | They propose a prototype-based representation alignment model for a cross-lingual named entity recognition task using labeled source language data. |
| Outcome: | The proposed model outperforms existing state-of-the-art methods in some challenging scenarios. |
Evaluation of Sentence Representations in Polish (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for learning sentence representations have been limited in low-resource languages such as Polish . |
| Approach: | They propose two new Polish datasets for evaluating sentence embeddings and evaluate eight different methods including Polish and multilingual models. |
| Outcome: | The proposed methods show strengths and weaknesses in Polish and multilingual models. |
Convolutional Neural Network for Universal Sentence Embeddings (C18-1)
Copied to clipboard
| Challenge: | Recent studies show that averaging word embeddings is effective for NLP but these models represent a sentence only in terms of features of words or uni-grams. |
| Approach: | They propose a CNN-based model that uses both features of words and n-grams to encode sentences. |
| Outcome: | The proposed model performs better than existing models in transfer learning setting and exceeds state of the art in supervised learning setting by initializing the parameters with the pre-trained sentence embeddings. |
Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing strategies to circumvent safety constraints face significant trade-offs between effectiveness and efficiency. |
| Approach: | They propose a framework that allows to infer model refusal behaviors without expensive parameter updates or training. |
| Outcome: | The proposed framework outperforms baselines in multiple safety-aligned open-source LLMs. |
Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels (2021.acl-long)
Copied to clipboard
| Challenge: | Existing resources for idiomaticity annotation only include ratings at type level . idioms such as noun compounds have been considered a challenge for NLP . |
| Approach: | They present a dataset with human annotations for 280 noun compounds in English and 180 in Portuguese at both type and token levels. |
| Outcome: | The proposed dataset shows that human annotations are not capturing idiomaticity as human annotation models. |
Insert or Attach: Taxonomy Completion via Box Embedding (2024.acl-long)
Copied to clipboard
| Challenge: | Existing taxonomy expansion methods embed concepts as vectors in Euclidean space, causing incorrectly model asymmetric relations. |
| Approach: | They propose to use box containment and center closeness to create geometric scorers that capture intrinsic relationships between concepts. |
| Outcome: | The proposed framework outperforms existing methods on four real-world datasets. |
Personalized Neural Embeddings for Collaborative Filtering with Text (N19-1)
Copied to clipboard
| Challenge: | Traditional CF approaches exploit user-item relations only and suffer from data sparsity issues. |
| Approach: | They develop a Personalized Neural Embedding framework to exploit both interactions and words seamlessly. |
| Outcome: | The proposed framework exploits both interactions and words seamlessly and predicts user preferences on items based on these embeddings. |
Semantic Frame Forecast (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work focused on predicting the immediate future of a story, such as one to a few sentences ahead. |
| Approach: | They propose a task that predicts the semantic frames that will occur in the next 10, 100, or even 1,000 sentences in a running story. |
| Outcome: | The proposed model outperforms random, prior, and replay baselines when the block size is over 150 sentences. |
Cross-Modal Discrete Representation Learning (2022.acl-long)
Copied to clipboard
| Challenge: | a new framework for learning representations from multimodal data is proposed . the proposed framework uses discretized embedding vectors to capture finer levels of granularity . |
| Approach: | They propose a self-supervised representation learning framework that captures finer levels of granularity across different modalities. |
| Outcome: | The proposed representation can capture finer levels of granularity across different modalities . it can be used on cross-modal retrieval tasks without direct supervision . |
Learned Incremental Representations for Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | a new syntactic representation that commits to syntakic choices is proposed for humans . we use a system that uses only incremental processing of a prefix to predict the word in a sentence . |
| Approach: | They propose a syntactic representation that commits to syntakic choices incrementally . they say the system can achieve 93.72 F1 on the Penn Treebank with as few as 5 bits per word . |
| Outcome: | The proposed representation achieves 93.72 F1 on the Penn Treebank with as few as 5 bits per word . the analysis of the representations shows they have interpretable features and deferred resolution of syntactic ambiguities. |
hyperdoc2vec: Distributed Representations of Hypertext Documents (P18-1)
Copied to clipboard
| Challenge: | Conventional text embedding methods suffer from information loss if directly adapted to hyper-documents. |
| Approach: | They propose an embedding approach for hyper-documents that incorporates four criteria to preserve necessary information for embeddable models. |
| Outcome: | The proposed model outperforms several existing models on two tasks in the academic domain. |
LaCoMSA: Language-Consistency Multilingual Self-Alignment with Latent Representation Rewarding (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing multilingual alignment methods mitigate these issues but rely on external supervision, such as translation systems or English-biased signal. |
| Approach: | They propose a preference optimization framework that leverages an LLM’s own latent representations as intrinsic supervision signals and rewards lower-resource language outputs based on their alignment with high-resourced (English) counterparts in the "semantic hub". |
| Outcome: | The proposed framework improves a Llama 3 8B model multilingual win rates by up to 6.8% absolute (55.0% relative) on X-AlpacaEval and achieves consistent gains across benchmarks and models. |
What’s in Your Embedding, And How It Predicts Task Performance (C18-1)
Copied to clipboard
| Challenge: | Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. |
| Approach: | They propose a method that quantifies interpretable characteristics of word vector neighborhoods and shows how they correlate with performance on 14 extrinsic and intrinsic task datasets. |
| Outcome: | The proposed approach enables multi-faceted evaluation, parameter search, and generally – a more principled, hypothesis-driven approach to development of distributional semantic representations. |
Multilingual AMR-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on generating text from structured data into English has focused on bridging the gap between structure and natural language (NL) and semantically underspecified input and fully specified output. |
| Approach: | They propose a multilingual approach that can decode into 21 different languages . they leverage advances in cross-lingual embeddings and pretraining to generate multilingual models . |
| Outcome: | The proposed model surpasses baselines that generate into one language in eighteen languages. |
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for gender bias mitigation for word embeddings are based on pre-trained word embeds . however, the assumption that the bias subspace is linear is untested . |
| Approach: | They propose a method to isolate gender bias in word embeddings using pre-trained word embeds. |
| Outcome: | The proposed method eliminates gender bias in word embeddings but assumes bias subspace is linear . the proposed method has some drawbacks, but it is a good one for a non-linear analysis. |
Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains (2020.emnlp-main)
Copied to clipboard
| Challenge: | Asymmetrical text matching is a fundamental problem in information retrieval and natural language processing. |
| Approach: | They propose a method that regularizes features vectors projected from different domains . WD-Match can be used to improve different text matching methods . |
| Outcome: | The proposed method outperforms existing methods and benchmarks on four datasets. |
Cross-Pair Text Representations for Answer Sentence Selection (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to textual entailment and question answering focus on intra-pair similarity . a simple lexical matching (marked with italics) is not enough to learn a model based on intrapair Qto-A similarities. |
| Approach: | They propose to compute scalar products representing similarity between members of different pairs instead of using a single vector for each pair. |
| Outcome: | The proposed approach outperforms more complex models based on neural networks. |
Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Bias (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that embedding models exhibit systematic positional and language biases when documents are longer and consist of multiple segments. |
| Approach: | They propose a permutation-based evaluation framework to quantify embedding biases . they propose an inference-time attention calibration method that redistributes attention more evenly across document positions . |
| Outcome: | The proposed framework reduces the positional and language biases in embedding models . the proposed framework improves the discoverability of later segments . |
Pareto Probing: Trading Off Accuracy for Complexity (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks are a pillar of modern NLP systems, but their inner workings are poorly understood. |
| Approach: | They propose a probe metric that reflects the trade-off between probe complexity and performance: the Pareto hypervolume. |
| Outcome: | The proposed probe metric conforms to accepted rankings among contextual representations, and is more complex than other probe tasks. |
Glyph2Vec: Learning Chinese Out-of-Vocabulary Word Embedding from Glyphs (2020.acl-main)
Copied to clipboard
| Challenge: | Chinese NLP applications that rely on large text often contain huge amounts of vocabulary which are sparse in corpus. |
| Approach: | They propose a multi-modal model that extracts visual features from Chinese word glyphs to expand current word embedding space without accessing any corpus. |
| Outcome: | The proposed model can embed words in Chinese without accessing corpus without a corpus. |
Story Trees: Representing Documents using Topological Persistence (2022.lrec-1)
Copied to clipboard
| Challenge: | Topological data analysis (TDA) focuses on the inherent shape of (spatial) data. |
| Approach: | They propose to use topological data analysis to represent document structure as story trees . story trees are hierarchical representations created from semantic vector representations of sentences . |
| Outcome: | The proposed methods can be used to extract summary summaries from news stories using story trees. |
Spectral Filters, Dark Signals, and Attention Sinks (2024.acl-long)
Copied to clipboard
| Challenge: | Recent work assigns a central role to the model's residual stream as the shared communication channel between model components. |
| Approach: | They propose a quantitative extension of the logit lens approach by partitioning the embedding and unembedding matrices into bands and spectral filters on intermediate representations. |
| Outcome: | The proposed model can suppress the tail end of the embedding spectrum, but it is not able to suppress large parts of the spectrum. |
Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to reduce dataset bias rely on spurious correlations and obstruct valid feature information while mitigating bias. |
| Approach: | They propose a representation normalization method which disentangles correlations between features of encoded sentences and a kernel approximation method which provides isotropic data distribution. |
| Outcome: | The proposed method eliminates the bias problem by providing isotropic data distribution while maintaining in-distribution accuracy. |
Learning a Reversible Embedding Mapping using Bi-Directional Manifold Alignment (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models that perform unidirectional mappings are unipolar and unintended . |
| Approach: | They propose a bi-directional mapping algorithm that learns a non-linear mapping between two manifolds by explicitly training it to be bijective. |
| Outcome: | The proposed model reduces the number of models by 50% and improves performance over unidirectional translation models. |
Learning Better Internal Structure of Words for Sequence Labeling (D18-1)
Copied to clipboard
| Challenge: | a gap exists between methods for learning representations of sentences and words . authors propose a convolutional neural architecture with no down-sampling for learning words based on character embeddings . |
| Approach: | They propose a funnel-shaped wide convolutional neural architecture with no down-sampling for learning words' internal structure. |
| Outcome: | The proposed model outperforms other character embedding models on six sequence labeling datasets. |
Entailment-Preserving First-order Logic Representations in Natural Language Entailment (2025.acl-long)
Copied to clipboard
| Challenge: | First-order logic (FOL) is often used to represent logical entailment, but determining natural language (NL) enanglement using FOL remains a challenge. |
| Approach: | They propose an Entailment-Preserving FOL representations task and a method which trains an NL-to-FOL translator by using the natural language entailment labels as verifiable rewards. |
| Outcome: | The proposed method achieves 1.8–2.7% improvement in EPR and 17.4–20.6% increase in E PR@16 compared to baselines in three datasets. |
Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Text embedding models are widely used in natural language processing but are often benchmarked on tasks that do not require understanding nuanced numerical information in text. |
| Approach: | They evaluate 13 widely used text embedding models and find they struggle to capture numerical details accurately. |
| Outcome: | The proposed models struggle to capture nuanced numerical details accurately, despite being benchmarked on tasks that do not require understanding nuance. |
Simple Algorithms For Sentiment Analysis On Sentiment Rich, Data Poor Domains. (C18-1)
Copied to clipboard
| Challenge: | Standard word embedding algorithms learn vector representations from large corpora of text documents in unsupervised fashion. |
| Approach: | They propose an algorithm that learns word embeddings jointly with a classifier . their algorithm leverages document label information to learn vector representations of words . |
| Outcome: | The proposed algorithm has superior performance on domains with limited data compared to other methods. |
CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies integrate word embeddings with cognitive features into neural models of natural language processing (NLP) but there are some issues in the use of cognitive features in NLP. |
| Approach: | They propose a cog-align approach that aligns textual and cognitive inputs to capture differences and commonalities. |
| Outcome: | The proposed model improves on three NLP tasks with multiple cognitive features over state-of-the-art models. |
XTRA: Cross-Lingual Topic Modeling with Topic and Representation Alignments (2025.findings-emnlp)
Copied to clipboard
| Challenge: | XTRA aims to uncover shared semantic themes across languages . previous methods have achieved improvements in topic diversity but struggle to ensure high topic coherence and consistent alignment across languages. |
| Approach: | a new framework unifies Bag-of-Words modeling with multilingual embeddings is proposed to address this problem . XTRA introduces two core components: (1) representation alignment and (2) topic alignment to enforce cross-lingual consistency. |
| Outcome: | XTRA outperforms baselines in topic coherence, diversity, and alignment quality on multilingual corpora. |
Reducing Gender Bias in Abusive Language Detection (D18-1)
Copied to clipboard
| Challenge: | Abusive language detection models tend to be biased toward identity words of a certain group of people . recent studies have raised concerns about the robustness of such systems . |
| Approach: | They propose to use debiased word embeddings, gender swap data augmentation to reduce model bias . they also propose to fine-tune models with a larger corpus to correct such bias if needed . |
| Outcome: | The proposed methods reduce model bias by 90-98% and can be extended to correct model bias in other scenarios. |
Learning multiview embeddings for assessing dementia (D18-1)
Copied to clipboard
| Challenge: | In 2017, 5.7 million Americans were living with Alzheimer's disease (AD), and the disease accounted for $11.4 billion in healthcare costs in the United States. |
| Approach: | They leverage the multiview nature of a small AD dataset to learn an embedding that captures different modes of cognitive impairment. |
| Outcome: | The proposed embeddings achieve an F1 score of 0.82 and a mean absolute error of 3.42 in the classification task and predicting clinical scores. |
Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Clinical notes of patient EHRs contain valuable information from healthcare professionals, but have been underutilized due to their difficult-to-understand contents and complex hierarchies. |
| Approach: | They propose to use clinical notes to learn more balanced knowledge from EHRs by assembling useful neutral words with rare keywords via note and taxonomy level hyperedges. |
| Outcome: | The proposed method can retain clinical semantic information by (1) frequent neutral words and (2) hierarchies with imbalanced distribution. |
Stock Embeddings Acquired from News Articles and Price History, and an Application to Portfolio Optimization (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that news articles can be leveraged to improve price prediction. |
| Approach: | They propose a method to encode the influence of news articles through a vector representation of stocks . they use a deep learning framework to acquire the vector representation using news articles and price history . |
| Outcome: | The proposed method can be applied to other financial problems besides price prediction. |
Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings (P19-1)
Copied to clipboard
| Challenge: | Traditional parallel corpus mining methods focus on the textual content instead of the size and quality of training data. |
| Approach: | They propose a method for machine translation based on multilingual sentence embeddings. |
| Outcome: | The proposed method outperforms the best published methods on the BUCC mining task and the UN reconstruction task by more than 10 F1 and 30 precision points. |
Probing for idiomaticity in vector space models (2021.eacl-main)
Copied to clipboard
| Challenge: | Contextualised word representation models are used to represent idiomaticity in language. |
| Approach: | They propose probing measures to assess if some of the expected linguistic properties of noun compounds are readily available in some standard and widely used representations. |
| Outcome: | The proposed models show that idiomaticity is not yet accurately represented by contextualised models. |
Tell Me What You Don’t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing (2025.findings-acl)
Copied to clipboard
Wenhao Liu, Siyu An, Junru Lu, Muling Wu, Tianlong Li, Xiaohua Wang, Changze Lv, Xiaoqing Zheng, Di Yin, Xing Sun, Xuanjing Huang
| Challenge: | Role-playing Agents (RPAs) struggle to recognize and respond to hard queries that conflict with their role-play knowledge. |
| Approach: | They propose a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model’s refusal accuracy. |
| Outcome: | The proposed model improves RPAs’ refusal ability of conflicting requests while maintaining their general role-playing capabilities. |
AMR Beyond the Sentence: the Multi-sentence AMR corpus (C18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is limited to capturing the semantics of individual sentences. |
| Approach: | They propose a corpus that annotates coreference and similar phenomena on top of existing AMRs. |
| Outcome: | The proposed corpus is compared with existing corpora on sentence-level semantics . it shows that it can be used for information extraction and question answering . |
Is Word Segmentation Necessary for Deep Learning of Chinese Representations? (P19-1)
Copied to clipboard
| Challenge: | Using word-based models, we compare word-oriented models with char-based ones . word-driven models are more vulnerable to data sparsity and the presence of out-of-vocabulary words . |
| Approach: | They benchmark word-based models with char-based model which does not involve word segmentation in four NLP benchmark tasks. |
| Outcome: | The proposed model outperforms char-based models in four NLP benchmark tasks. |
On the Compositionality Prediction of Noun Phrases using Poincaré Embeddings (P19-1)
Copied to clipboard
| Challenge: | idiomatic phrases have a non-compositional meaning, meanings of which can be derived from constituents and their grammatical relations. |
| Approach: | They propose to combine hierarchical and distributional information to blend hierarchic and distribution-based hierarchies to detect compositionality for noun phrases. |
| Outcome: | The proposed technique achieves significant improvements over state-of-the-art models based on distributional information and a weighted average of the distributional similarity and p-like function. |
Robust Representation Learning of Biomedical Names (P19-1)
Copied to clipboard
| Challenge: | Biomedical concepts are often mentioned in medical documents under different name variations. |
| Approach: | They propose a framework for learning robust representations of biomedical names and terms . they encode contextual meaning, conceptual meaning, and similarity between synonyms . |
| Outcome: | The proposed framework outperforms baselines on retrieval, similarity and relatedness benchmarks. |
SD-E2: Semantic Exploration for Reasoning Under Token Budgets (2026.findings-eacl)
Copied to clipboard
| Challenge: | Small language models struggle with complex reasoning because exploration is expensive under tight compute budgets. |
| Approach: | They propose a framework that makes exploration explicit by optimizing semantic diversity in generated reasoning trajectories. |
| Outcome: | The proposed framework surpasses Qwen2.5-3B-Instruct and strong GRPO baselines on GSM8K and improves on the harder AIME benchmark to 13.28% vs. base 6.74%. |
Segmentation-free compositional n-gram embedding (N19-1)
Copied to clipboard
| Challenge: | Existing word embedding models depend on word segmentation, but this method is difficult when corpora written in noisy or unsegmented languages. |
| Approach: | They propose a new method that models words, phrases and sentences seamlessly without word segmentation. |
| Outcome: | The proposed method is very effective for noisy corpora written in unsegmented languages such as Chinese and Japanese. |
Invertible Tree Embeddings using a Cryptographic Role Embedding Scheme (2020.coling-main)
Copied to clipboard
| Challenge: | Unlike previous attempts, this method does not come at the cost of intractable representation size; it works well when there is sufficient randomness in the representation scheme for simple data and providing an upper bound on its error. |
| Approach: | They propose a method for embedding trees in a vector space based on Tensor-Product Representations (TPRs) that allows for inversion: the retrieval of the original tree structure and nodes from the vectorial embeddment. |
| Outcome: | The proposed method can provide invertibility with error 1% that previous methods would require 8.6 1057 dimensions to represent. |
H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to align pre-trained LLMs with instructions for one property are difficult to fine-tune. |
| Approach: | They propose a mixture-of-experts-based fusion mechanism that models alignment as a controllable drift within the subspace, guided by a drift-regularization loss to balance competing alignment dimensions. |
| Outcome: | Extensive evaluations of three benchmark datasets show that H3Fusion outperforms each individually aligned model by 11.37% and provides stronger robustness compared to the state-of-the-art LLM ensemble approaches by 13.77% and model-merging approaches by 6.18 %. |
Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity (2022.naacl-main)
Copied to clipboard
| Challenge: | Using co-citations, we can train a model that matches aspects of papers to document level similarity. |
| Approach: | They propose a model that matches fine-grained aspects of papers and aggregates them into a document level similarity model using a naturally-occurring source of supervision: co-citations. |
| Outcome: | The proposed model improves performance on document similarity tasks in four datasets and achieves competitive results. |
Towards Better Context-aware Lexical Semantics:Adjusting Contextualized Representations through Static Anchors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has shown that contextualized models generate dynamic embeddings for words in context, but static embedds are often overlooked in this trend towards contextualized modeling. |
| Approach: | They propose a method that learns a transformation through static anchors and requires only another pre-trained model. |
| Outcome: | The proposed method improves a range of benchmark tasks that test contextual variations of meaning across different usages of a word and across different words as they are used in context. |
Investigating Word-Class Distributions in Word Vector Spaces (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies have been successful in representing the meaning of a word with a vector in a continuous vector space, but little attention has been paid to the distribution of words belonging to a certain word class in . word vector spaces are useful for a range of natural language processing tasks, including selectional preference acquisition and entity set expansion. |
| Approach: | They investigated the distribution of word vectors belonging to a certain word class in a pre-trained word vector space and compared their models to validate their assumptions. |
| Outcome: | The proposed model fails to estimate how likely a word in the vector space is a member of a given word class, and the geometry of the distribution and existence of subgroups will have limited impact. |
One Word, Two Sides: Traces of Stance in Contextualized Word Representations (2022.coling-1)
Copied to clipboard
| Challenge: | a Lexical Semantic Change study examines the way we use words . it focuses on the use of words by people who disagree on a particular topic . |
| Approach: | They examine whether word embeddings reflect the way we use words . they use BERT embeddables from datasets with stance annotations to examine this question . |
| Outcome: | The results show that people with opposing stances use different words when talking about a topic . the results are not related to studies that investigate the usage of specific words across different viewpoints. |
HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing taxonomies have limited coverage due to expensive manual curation process. |
| Approach: | They propose an algorithm that expands existing taxonomies to preserve their structure in a more expressive hyperbolic embedding space and learns to represent concepts and their relations with a hyperbolical Graph Neural Network. |
| Outcome: | The proposed algorithm outperforms baseline models with representation learning in a Euclidean feature space and achieves state-of-the-art performance on the taxonomy expansion benchmarks. |
A Neural Network Architecture for Program Understanding Inspired by Human Behaviors (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies for understanding programs do not take human behaviors as reference. |
| Approach: | They propose a graph neural network model that takes human behaviors as reference in understanding programs. |
| Outcome: | The proposed model performs better on code summarization and code clone detection tasks. |
Bayesian Hierarchical Words Representation Learning (2020.acl-main)
Copied to clipboard
| Challenge: | Using hierarchical priors, BHWR improves word representations by incorporating word semantic taxonomy. |
| Approach: | They propose a Bayesian Hierarchical Words Representation (BHWR) learning algorithm that integrates hierarchical priors and word semantic taxonomy to improve representations. |
| Outcome: | The proposed model performs better on rare words and on linguistic datasets than other methods. |
Bridging the Defined and the Defining: Exploiting Implicit Lexical Semantic Relations in Definition Modeling (D19-1)
Copied to clipboard
| Challenge: | Existing definition modeling methods do not utilize lexical semantic relations between defined words and defining words. |
| Approach: | They propose definition modeling methods that use lexical semantic relations . they use unsupervised pattern-based word-pair embeddings that represent semantic relations of word pairs . |
| Outcome: | The proposed methods improve definition generation and learning embeddings from definitions. |
Adapting General-Purpose Embedding Models to Private Datasets Using Keyword-based Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Text embedding models show strong performance on generic benchmarks, but their effectiveness diminishes when applied to private datasets. |
| Approach: | They propose a method for adapting general-purpose text embedding models to private datasets . they construct supervisory signals from the ranking of keyword-based retrieval results . |
| Outcome: | The proposed method improves retrieval performance across domains, datasets, and models. |
CILex: An Investigation of Context Information for Lexical Substitution Methods (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for lexical substitution rely on manually curated lexicals and contextual word embedding models. |
| Approach: | They propose a method that uses contextual sentence embeddings to generate substitutes for a target word given a context and a model that captures additional context information complimenting contextual word embedders. |
| Outcome: | The proposed method is state-of-the-art on the widely used LS07 and CoInCo datasets with P@1 scores of 55.96% and 57.25% for lexical substitution. |
pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference (N19-1)
Copied to clipboard
| Challenge: | Existing inference models that rely heavily on unsupervised single-word embeddings struggle to learn implied relationships between pairs of words. |
| Approach: | They propose to use word embeddings to learn and use background knowledge about implied relationships between words that are crucial for cross-sentence inference problems. |
| Outcome: | The proposed models gain 2.7% on the recently released SQuAD 2.0 and 1.3% on MultiNLI, and 8.8% on the adversarial SQu AD datasets. |
A Deep Decomposable Model for Disentangling Syntax and Semantics in Sentence Representation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in disentanglement work on coarse levels in the disenanglement of closely related properties, such as syntax and semantics in human languages. |
| Approach: | They propose a deep decomposable model based on VAE to disentangle syntax and semantics by using total correlation penalties on KL divergences. |
| Outcome: | The proposed model significantly improves the disentanglement quality between syntactic and semantic representations for semantic similarity tasks and syntaktic similarity task. |
Adaptive Compression of Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Distributed representations of words have been an indispensable component for natural language processing (NLP) tasks. |
| Approach: | They propose a method that uses a code-book approach to represent words as discrete codes such as (8, 5, 2, 4). |
| Outcome: | The proposed method makes the highly compressed word embeddings without hurting the task accuracy. |
SMS Spam Detection Through Skip-gram Embeddings and Shallow Networks (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting SMS spam include CNN, Recurrent Neural Networks and CNN. |
| Approach: | They propose a technique for embedding SMS messages into vector spaces suitable for spam detection. |
| Outcome: | The proposed method is competitive with state-of-the-art methods on a UCI Spam Collection dataset. |
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text (D19-1)
Copied to clipboard
| Challenge: | Variational Autoencoders are powerful language models and effective representation learning frameworks. |
| Approach: | They propose a fix for posterior collapse which improves held-out likelihood, reconstruction and latent representation learning . |
| Outcome: | The proposed fix significantly improves held-out likelihood, reconstruction, and latent representation learning compared with previous state-of-the-art methods. |
Development of a Japanese Personality Dictionary based on Psychological Methods (2020.lrec-1)
Copied to clipboard
| Challenge: | a new approach to constructing a personality dictionary with psychological evidence is needed . we use abstract terms such as "sociable person" or "kind" to describe ourselves or others . |
| Approach: | They propose a Japanese personality dictionary with weights for Big Five traits . they collect personality words and use word embeddings to construct the dictionary . |
| Outcome: | The proposed approach is the first to have psychological evidence tolerant to NLP standards. |
Diachronic Sense Modeling with Deep Contextualized Word Embeddings: An Ecological View (P19-1)
Copied to clipboard
| Challenge: | Existing word embeddings only assign one vector to a word for a time period, thus they face the meaning conflation deficiency. |
| Approach: | They propose a sense representation and tracking framework based on deep contextualized embeddings that can be used to answer what and when the word meaning changes. |
| Outcome: | The proposed framework is effective in representing fine-grained word senses, and brings a significant improvement in word change detection task. |
Conditional Dichotomy Quantification via Geometric Embedding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods that rely on semantic similarity fail to capture the nuanced oppositional dynamics essential for these applications. |
| Approach: | They propose a task that formalizes the measurement of conditional dichotomy by using a dichotomian framework. |
| Outcome: | The proposed framework provides carefully constructed datasets covering debate, defeasible inference, and causal reasoning scenarios. |
On the Distribution of Deep Clausal Embeddings: A Large Cross-linguistic Study (P19-1)
Copied to clipboard
| Challenge: | Empirical evidence on the prevalence and limits of embeddings has been based on either laboratory setups or corpus data of relatively limited size. |
| Approach: | They use large, dependency-parsed corpora to capture clausal embedding through dependency graphs and assess their distribution. |
| Outcome: | The results show that there is no evidence for hard constraints on embedding depth . they also show that sentences with many embeddable clauses do not display a bias towards less deep embedded sentences. |
Methods for Numeracy-Preserving Word Embeddings (2020.emnlp-main)
Copied to clipboard
Dhanasekar Sundararaman, Shijing Si, Vivek Subramanian, Guoyin Wang, Devamanyu Hazarika, Lawrence Carin
| Challenge: | Word embedding models capture semantic relationships between words but fail to capture numerical properties associated with numbers. |
| Approach: | They propose a method to assign and learn embeddings for numbers using word embedders. |
| Outcome: | The proposed model outperforms pre-trained word embedding models across multiple examples of two tasks. |
Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval models based on dense representations show better performance than sparse representations. |
| Approach: | They propose a method to mimic the queries to each of the documents by an iterative clustering process and represent the documents using multiple pseudo queries. |
| Outcome: | The proposed model achieves state-of-the-art results on a large dataset while remaining high efficiency. |
Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations (2023.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition models rely on domain-specific dictionaries provided by experts . however, such dictionary sets are infeasible in many domains where they do not exist . |
| Approach: | They propose a framework that generates NER datasets with high-coverage pseudo-dictionaries . phrase retrieval models are used to retrieve popular entities rather than rare ones . |
| Outcome: | The proposed framework outperforms the previous best model by an average F1 score of 4.7 across five NER benchmark datasets. |
SEA-BED: How Do Embedding Models Represent Southeast Asian Languages? (2026.acl-long)
Copied to clipboard
Wuttikorn Ponwitayarat, Peerat Limkonchotiwat, Raymond Ng, Jann Railey Montalan, Thura Aung, Jian Gang Ngui, Yosephine Susanto, William Chandra Tjhi, Panuthep Tasawong, Erik Cambria, Ekapol Chuangsuwanich, Sarana Nutanong
| Challenge: | SEA-BED examines how multilingual text embeddings perform across tasks and languages . performance gaps arise from data coverage, training objectives, and architectural design, authors say . |
| Approach: | They propose a large-scale benchmark covering 10 SEA languages and diverse embedding tasks. |
| Outcome: | The proposed model performs poorly across languages and tasks, but language-task analyses reveal inconsistencies . the results suggest that performance gaps arise from limitations in data coverage, training objectives, and architectural design. |
RepEval: Effective Text Evaluation with LLM Representation (2024.emnlp-main)
Copied to clipboard
Shuqian Sheng, Yi Xu, Tianhang Zhang, Zanwei Shen, Luoyi Fu, Jiaxin Ding, Lei Zhou, Xiaoying Gan, Xinbing Wang, Chenghu Zhou
| Challenge: | Traditional metrics for automatic text evaluation are tailored to specific tasks, while LLM-based evaluation metrics are costly. |
| Approach: | They propose a metric that leverages projections of LLM representations for evaluation. |
| Outcome: | The proposed metric exhibits higher correlation with human judgments than previous methods on 14 datasets. |
DirectProbe: Studying Representations without Classifiers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches for probing opaque representations often use training classifiers and use the accuracy, mutual information, or complexity as a proxy for the representation’s goodness. |
| Approach: | They propose a heuristic that directly studies the geometry of a representation by building upon the notion of 'version space' they argue that doing so can be unreliable because different representations may need different classifiers . |
| Outcome: | Experiments with linguistic tasks and contextualized embeddings show that even without training classifiers, DirectProbe can shine lights on how an embeddable space represents labels and anticipate the classifier performance for the representation. |
Bootstrapped Unsupervised Sentence Representation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to learn sentence representations rely on quality labeled data. |
| Approach: | They propose a Siamese Network which maximizes similarity between two augmented views of each sentence. |
| Outcome: | The proposed method outperforms state-of-the-art methods on STS and classification tasks. |
Learning Transferable Feature Representations Using Neural Networks (P19-1)
Copied to clipboard
| Challenge: | Traditional domain adaptation algorithms learn common representations which suffer from transfer loss when the source specific characteristics detract their ability to represent the target data. |
| Approach: | They propose to segregate source specific representation from the common representation and use it to learn a two-part representation which captures source specific characteristics while the second part captures the truly common representation. |
| Outcome: | The proposed representation outperforms existing learning algorithms on the source learning as well as cross-domain tasks on multiple datasets. |
Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Exemplar-Guided Paraphrase Generation (EGPG) aims to generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. |
| Approach: | They propose a method to generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. |
| Outcome: | The proposed method can generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. |
A Structural Probe for Finding Syntax in Word Representations (N19-1)
Copied to clipboard
| Challenge: | Existing methods for detecting syntactic knowledge do not test whether syntax trees are embedded in a linear transformation of a neural network’s word representation space. |
| Approach: | They propose a structural probe which evaluates whether syntax trees are embedded in a linear transformation of a neural network’s word representation space. |
| Outcome: | The proposed model shows that entire syntax trees are embedded in deep models’ vector geometry. |
Non-Linearity in Mapping Based Cross-Lingual Word Embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on cross-lingual word embeddings rely on linear mappings, but this assumption is not true for all language pairs. |
| Approach: | They propose a non-linear mapping approach which can find non-linesar relationships between languages by kernel Canonical Correlation Analysis. |
| Outcome: | The proposed approach improves on five language pairs on supervised and self-learning scenarios. |
DisSent: Learning Sentence Representations from Explicit Discourse Relations (P19-1)
Copied to clipboard
| Challenge: | Existing models train on vast amounts of text or require costly, manually curated datasets. |
| Approach: | They propose to leverage the discourse relations between sentences to curate a high quality sentence relation task by leveraging explicit discourse relations. |
| Outcome: | The proposed model can be used to learn the meaning of two sentences in a bidirectional LSTM sentence encoder. |
Unsupervised Concept Representation Learning for Length-Varying Text Similarity (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing document similarity approaches suffer from the information gap caused by context and vocabulary mismatches when comparing varying-length texts. |
| Approach: | They propose an unsupervised concept representation learning approach to address this issue . they propose a concept-based document matching method to leverage recognition of local phrase features . |
| Outcome: | The proposed method achieves a better F1 score than baseline models on real-world data sets. |
A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for learning entity embeddings from text descriptions leave it to downstream applications to identify these different facets and to select the most relevant ones. |
| Approach: | They propose a model that instead learns several vectors for each entity, each of which captures a different aspect of the considered domain. |
| Outcome: | The proposed model learns several vectors for each entity, each of which intuitively captures a different aspect of the considered domain. |
Context-aware Embedding for Targeted Aspect-based Sentiment Analysis (P19-1)
Copied to clipboard
| Challenge: | Existing methods do not specifically pre-train reasonable embeddings for targets and aspects in TABSA. |
| Approach: | They propose to refine the embeddings of targets and aspects using a sparse coefficient vector . this allows the embeds to be refined from highly correlative words instead of context-independent vectors . |
| Outcome: | Experiments show that the proposed method improves on two benchmark datasets. |
An Empirical Study of Span Representations in Argumentation Structure Parsing (P19-1)
Copied to clipboard
Tatsuki Kuribayashi, Hiroki Ouchi, Naoya Inoue, Paul Reisert, Toshinori Miyoshi, Jun Suzuki, Kentaro Inui
| Challenge: | Argumentation structure parsing (ASP) is a task of identifying argumentation structures in argumentative text. |
| Approach: | They propose to exploit neural network-based span representations for ASP to improve performance . they also propose task-dependent extensions for a parser that can be used to parse arguments . |
| Outcome: | The proposed model outperforms neural network-based approaches for argumentation structure parsing (ASP) it also provides some challenging types of instances to be parsed. |
Interpreting Character Embeddings With Perceptual Representations: The Case of Shape, Sound, and Color (2022.acl-long)
Copied to clipboard
| Challenge: | Character-level information is included in many NLP models, but evaluating the information encoded in character embeddings is an open issue. |
| Approach: | They propose to use shape, sound, and color embeddings to evaluate the information encoded in character representations in five languages to perform cross-lingual analysis. |
| Outcome: | The proposed classifiers evaluate phonological information encoded in character embeddings and LSTM models. |
Break it Down into BTS: Basic, Tiniest Subword Units for Korean (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing word embeddings for Korean use the internal structure of words with subword information to improve the quality of word representations. |
| Approach: | They introduce Basic, Tiniest Subword (BTS) units for Korean language that are inspired by Hangeul, the Korean writing system. |
| Outcome: | The proposed framework outperforms the state-of-the-art Korean word embedding by 11.8% on all intrinsic and extrinsic tasks. |
A Deep Neural Information Fusion Architecture for Textual Network Embeddings (D19-1)
Copied to clipboard
| Challenge: | Textual network embeddings aim to learn a low-dimensional representation for every node in the network while seeking to retain the original network information. |
| Approach: | They propose a deep neural architecture to fuse the two kinds of informations into one representation. |
| Outcome: | The proposed model outperforms the comparing methods on all three datasets. |
You Shall Know a User by the Company It Keeps: Dynamic Representations for Social Media Users in NLP (D19-1)
Copied to clipboard
| Challenge: | Current approaches to social media modelling ignore the fact that an individual may be part of several communities which are not equally relevant in all communicative situations. |
| Approach: | They propose a model that captures the sociological phenomenon of homophily and combines it with linguistic information to make a prediction. |
| Outcome: | The proposed model significantly outperforms existing models on three different tasks and is compared with other models. |
A Text Embedding Model with Contrastive Example Mining for Point-of-Interest Geocoding (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on coarse-grained locations, but we focus on fine-grain POIs, which have many candidates with similar names. |
| Approach: | They develop a text embedding-based geocoding model and investigate (1) entry encoding representations and (2) hard negative mining approaches suitable for enhancing the model’s disambiguation ability. |
| Outcome: | The proposed model significantly improves its disambiguation ability and entry encoding representations. |
Towards Understanding the Relation between Gestures and Language (2022.coling-1)
Copied to clipboard
| Challenge: | a new study explores the relationship between gestures and language . we use contrastive learning to learn gesture embeddings . |
| Approach: | They adapt a semi-supervised multimodal model to learn gesture embeddings using Ted talks . they show gestures are predictive of the native language of the speaker . |
| Outcome: | The proposed model learns gesture embeddings from a multimodal dataset . it shows that gesture embeds are predictive of the native language of the speaker . |
News2vec: News Network Embedding with Subnode Information (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to embed news as vectors do not integrate features and inter-textual knowledge of news. |
| Approach: | They propose a model that integrates news features and inter-textual knowledge into a dense vector representation. |
| Outcome: | The proposed model can be used to represent news as a dense vector . it is compared with existing models on stock movement prediction and news recommendation tasks . |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |
Extracting Conceptual Spaces from LLMs Using Prototype Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Conceptual spaces represent entities and concepts using cognitively meaningful dimensions . practical methods for extracting conceptual spaces are currently lacking . |
| Approach: | They propose a strategy in which features are encoded by embedding a description of a corresponding prototype. |
| Outcome: | The proposed approach is highly effective. |
Neural News Recommendation with Heterogeneous User Behavior (D19-1)
Copied to clipboard
| Challenge: | Existing news recommendation methods rely on news click history to model user interest, but data sparsity is a problem . other kinds of user behaviors such as webpage browsing and search queries can provide useful clues of users’ news reading interest. |
| Approach: | They propose to exploit heterogeneous user behaviors to learn news representations from their titles via CNN networks and apply attention networks to select important words. |
| Outcome: | The proposed approach exploits heterogeneous user behaviors on a real-world dataset. |
Reviews Meet Graphs: Enhancing User and Item Representations for Recommendation with Hierarchical Attentive Graph Neural Network (D19-1)
Copied to clipboard
| Challenge: | Existing methods to learn user and item representations from review texts do not take into account the user-user and item-item relatedness of the user. |
| Approach: | They propose to use review content and user-item graphs to integrate them as different views. |
| Outcome: | The proposed approach can learn user and item representations from review content and user-item graphs. |
Event Representation Learning Enhanced with External Commonsense Knowledge (D19-1)
Copied to clipboard
| Challenge: | Existing methods to learn event representations from text lack commonsense knowledge about the intents and emotions of event participants. |
| Approach: | They propose to leverage external commonsense knowledge about the intent and sentiment of the event to learn distributed representations for structured events from text. |
| Outcome: | The proposed model improves on hard similarity tasks and yields more precise inferences on subsequent events under given contexts. |
CausalCite: A Causal Formulation of Paper Citations (2024.findings-acl)
Copied to clipboard
Ishan Agrawal, Zhijing Jin, Ehsan Mokhtarian, Siyuan Guo, Yuen Chen, Mrinmaya Sachan, Bernhard Schölkopf
| Challenge: | citation counts are often criticized for failing to accurately reflect the true impact of a paper. |
| Approach: | They propose a method to measure the impact of a paper on follow-up papers by comparing similar papers by cosine similarity. |
| Outcome: | The proposed method is based on a new causal inference method, TextMatch. |
Learning Disentangled Representations of Texts with Application to Biomedical Abstracts (D18-1)
Copied to clipboard
| Challenge: | a method for learning disentangled representations of texts that encode distinct and complementary aspects is proposed . a classic problem in distributed representation learning is that it is difficult to determine what information individual dimensions encode. |
| Approach: | They propose a method for learning disentangled representations of texts that encode distinct and complementary aspects by a adversarial objective based on the (dis)similarity between triplets of documents with respect to specific aspects. |
| Outcome: | The proposed method can be used to perform aspect-specific retrieval on biomedical abstracts. |
Multilingual Culture-Independent Word Analogy Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | In text processing, deep neural networks use word embeddings as an input. |
| Approach: | They propose to use benchmark datasets to compare the quality of word embeddings in text processing . they use a word analogy task in Croatian, English, Estonian, Finnish, Latvian, Lithuanian, Russian, Slovenian, and Swedish . |
| Outcome: | The proposed datasets are culturally independent and cross-lingual for the languages used. |
Towards Transferable Personality Representation Learning based on Triplet Comparisons and Its Applications (2025.emnlp-main)
Copied to clipboard
Kai Tang, Rui Wang, Renyu Zhu, Minmin Lin, Xiao Ding, Tangjie Lv, Changjie Fan, Runze Wu, Haobo Wang
| Challenge: | Existing methods for personality analysis treat corpus as a single unit for classification, but this approach presents several challenges. |
| Approach: | They propose a task paradigm for text-based personality representation learning that uses a triplet personality trend comparison dataset to learn single-sentence personality embeddings with desirable metric properties. |
| Outcome: | The proposed model significantly boosts performance across various applications, including personality detection, personality retrieval, and emotion translation prediction. |
A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of textual data are difficult to implement and suffer from the degeneracy of other losses in multi-class scenarios. |
| Approach: | They propose a variational upper bound to the mutual information between an attribute and the latent code of an encoder that controls the approximation error. |
| Outcome: | The proposed method is superior on fair classification and on textual style transfer tasks. |
VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for definition modeling combine distributional and lexical semantics in an implicit rather than direct way. |
| Approach: | They propose a model that introduces a continuous latent variable to model the relationship between a phrase and its definition. |
| Outcome: | The proposed model achieves state-of-the-art performance on four challenging benchmarks and the first non-English corpus. |
Learning Gender-Neutral Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Word embeddings trained on human-generated corpora inherit strong gender stereotypes . prior studies show such embeddables exhibit social biases, such as gender stereotype . |
| Approach: | They propose a method to preserve gender information in certain dimensions of word vectors . they propose GN-GloVe, which is a gender-neutral variant of the word embedding model . |
| Outcome: | The proposed method preserves gender information in certain dimensions of word vectors while compelling other dimensions to be free of gender influence. |
Multiplex Word Embeddings for Selectional Preference Acquisition (D19-1)
Copied to clipboard
| Challenge: | Existing word embeddings are limited in their ability to represent fixed vectors . instead, they incorporate relational dependencies of different words into their embeddables - a limitation that is addressed by a multiplex model . |
| Approach: | They propose a word embedding model which incorporates relational dependencies of different words into their embeddables. |
| Outcome: | The proposed model can be easily extended according to various relations among words. |
A Hmong Corpus with Elaborate Expression Annotations (2022.lrec-1)
Copied to clipboard
| Challenge: | SCH is the first substantial corpus to be annotated for elaborate expressions . a plurality of speakers are located in China, but many Hmong speakers left Laos as refugees . |
| Approach: | They describe the first publicly available corpus of Hmong, a minority language of China, Vietnam, Laos, Thailand, and various countries in Europe and the Americas. |
| Outcome: | The first publicly available corpus of Hmong is scraped from a long-running Usenet newsgroup . it is the first substantial corpus to be annotated for elaborate expressions . |
All That Glitters is Not Gold: A Gold Standard of Adjective-Noun Collocations for German (2020.lrec-1)
Copied to clipboard
| Challenge: | Using the GerCo dataset, we identify adjective-noun collocations in German and compare them with statistical associations measures. |
| Approach: | They present a GerCo dataset of adjective-noun collocations for German, such as alter Freund ‘old friend’ and tiefe Liebe ‘deep love’. |
| Outcome: | The GerCo dataset contains 4,732 positive and negative instances of collocations and covers all 16 semantic classes of adjectives defined in the German wordnet GermaNet. |
Variants of Vector Space Reductions for Predicting the Compositionality of English Noun Compounds (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to predict the degree of compositionality of noun compounds are based on comparing compounds and their constituents within a vector space and using distributional similarity as a proxy to predict their degree of semantic relatedness. |
| Approach: | They propose to use distributional similarity as a proxy to predict the semantic relatedness between the compounds and their constituents as the compound’s degree of compositionality. |
| Outcome: | The proposed methods are most successful and stable in terms of dimensionality and part-of-speech reductions. |
An Element-aware Multi-representation Model for Law Article Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that using law articles as external knowledge can improve the performance of the Legal Judgment Prediction. |
| Approach: | They propose a Law Article Element-aware Multi-representation Model which makes full use of law article information and can be used for multi-label samples. |
| Outcome: | The proposed model improves the accuracy of 5.84%, macro F1 of 6.42%, and micro F1 by 4.28% compared with baseline models like TopJudge. |
COSMOS: Experimental and Comparative Studies of Concept Representations in Schoolchildren (2022.lrec-1)
Copied to clipboard
| Challenge: | COSMOS is a multidisciplinary research project investigating schoolchildren’s beliefs and representations of specific concepts under control variables (age, gender, language spoken at home). |
| Approach: | They present a lexical study of seven concepts in a french school . they use a word-level lexicon to examine their representations under control variables . |
| Outcome: | The results of the study show that children's linguistic proficiency and lexical diversity increase with age, and that gender and age influence lexicality. |
Empirical Linguistic Study of Sentence Embeddings (P19-1)
Copied to clipboard
| Challenge: | a new method of analysing sentence embeddings shows that linguistic information is retained in the vector representations of sentences. |
| Approach: | They propose a method of analysing the content of sentence embeddings based on probing tasks and contrasting languages. |
| Outcome: | The proposed method is based on probing tasks and classification datasets for two contrasting languages. |
Multimodal Clickbait Detection by De-confounding Biases Using Causal Representation Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new method to detect clickbait posts on the Web is needed to detect such posts. |
| Approach: | They propose a method to detect clickbait posts on the Web using latent factors . they use features in multiple modalities to characterize the posts and causal inference to eliminate noise . |
| Outcome: | The proposed method can detect clickbait posts on popular social media platforms with good generalization ability. |
CARER - ClinicAl Reasoning-Enhanced Representation for Temporal Health Risk Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing deep learning methods require large datasets to achieve high generalizability. |
| Approach: | They propose a framework that enhances deep learning models with clinical rationales derived from medically proficient Large Language Models. |
| Outcome: | The proposed framework outperforms state-of-the-art models on two tasks using two popular EHR datasets by up to 11.2%. |
Time-Aware Word Embeddings for Three Lebanese News Archives (2020.lrec-1)
Copied to clipboard
| Challenge: | a large corpus of newspaper archives has been generated, but historians have struggled to analyze it manually for decades. |
| Approach: | They propose to train word embeddings from three large Lebanese news archives, which collectively consist of 609,386 scanned newspaper images and span 151 years. |
| Outcome: | The embeddings are trained using a Google Tesseract 4.0 OCR engine and a benchmark of analogy tasks to evaluate their accuracy. |
GGP: Glossary Guided Post-processing for Word Embedding Learning (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing word embedding models require much training time and domain knowledge to improve. |
| Approach: | They propose a GGP-based word embedding model that incorporates the glossary and learns sense representations. |
| Outcome: | The proposed model outperforms existing models on topical/functional similarity datasets by 4.1% and 7%. |
Optimal Transport-based Alignment of Learned Character Representations for String Similarity (P19-1)
Copied to clipboard
| Challenge: | String similarity models are crucial for record linkage, data integration, search and entity resolution systems. |
| Approach: | They propose a model that encodes the characters of each string, aligns the encodings using Sinkhorn Iteration and scores the alignment with a convolutional neural network. |
| Outcome: | The proposed model outperforms state-of-the-art and classical similarity models on four of the five datasets and improves performance by applying it to cross-document coreference. |
Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for word embeddings have been used to model semantic relations with word embeds. |
| Approach: | They propose a method that leverages contextual embeddings for diachronic semantic shift detection by generating time specific word representations from BERT embedds. |
| Outcome: | The proposed method performs comparable to the current state-of-the-art without time consuming domain adaptation on large corpora. |
Detection of Reading Absorption in User-Generated Book Reviews: Resources Creation and Evaluation (2020.lrec-1)
Copied to clipboard
Piroska Lendvai, Sándor Darányi, Christian Geng, Moniek Kuijpers, Oier Lopez de Lacalle, Jean-Christophe Mensonides, Simone Rebora, Uwe Reichel
| Challenge: | a new study aims to detect how and when readers are experiencing engagement with a literary work . empirical literary studies and language technology are used to investigate reading absorption . |
| Approach: | They annotated user-generated book reviews with reading absorption categories . they then performed supervised binary classification of the mental state of absorption . |
| Outcome: | The proposed corpus of user-generated reviews is compared with machine learning models and a benchmark corpus. |
Learning distributed sentence vectors with bi-directional 3D convolutions (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods that render words or characters into images separately, but instead use text's visual features as input, we use 3-dimensional convolutions to learn distributed sentence representation. |
| Approach: | They propose to use text's visual features as input to learn distributed sentence representation using 3-dimensional sentence tensors and multiple 3-dimensional convolutions with different lengths are applied to the sentence . |
| Outcome: | The proposed model performs well on several downstream natural language processing tasks. |
Extracting Event Temporal Relations via Hyperbolic Geometry (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent neural approaches to event temporal relation extraction map events to embeddings in the Euclidean space and train a classifier to detect temporal relations between event pairs. |
| Approach: | They propose to embed events into hyperbolic spaces to model hierarchical structures . they propose to use hyperbolical embeddings to directly infer event relations . |
| Outcome: | The proposed architecture is based on two approaches to encode events and their temporal relations in hyperbolic spaces. |
An Attentive Fine-Grained Entity Typing Model with Latent Type Representation (D19-1)
Copied to clipboard
| Challenge: | Existing fine-grained entity typing models are criticized for label independence assumption . |
| Approach: | They propose a fine-grained entity typing model with a new attention mechanism and a hybrid type classifier to exploit type inter-dependency with latent type representation. |
| Outcome: | The proposed model significantly advances the state-of-the-art on fine-grained entity typing. |
Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that detect semantically shifted words do not account for its evolution through time. |
| Approach: | They propose three variants of sequential models for detecting semantically shifted words . they demonstrate that temporal modelling of word representations yields a clear-cut advantage . |
| Outcome: | The proposed models account for the changes in word representations over time. |
A Closer Look on Unsupervised Cross-lingual Word Embeddings Mapping (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for word embeddings are limited to a single, unannotated corpus, which means that word representations with similar meaning in distinct languages can be very different. |
| Approach: | They propose an unsupervised method for cross-lingual word embedding mapping that uses stochastic initialization and isometric initialization to verify the method's robustness. |
| Outcome: | The proposed method is robust on different embedding representations and new language pairs, particularly those involving Slavic languages like Polish or Czech. |
Sentiment Analysis of Tweets using Heterogeneous Multi-layer Network Representation and Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to handle sentiment analysis of tweets are inadequate due to various characteristics such as under-specificity, noise, and multilingual content. |
| Approach: | They propose a multi-layer network-based representation of tweets to generate multiple representations of a tweet and classify them using a neural-based early fusion approach. |
| Outcome: | The proposed method can address the problem of under-specificity, noisy text, and multilingual content present in a tweet and provides better representations than the text-based counterparts. |
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work has introduced Abstract Meaning Representation (AMR) for Document-level Event Argument Extraction (Doc-level EAE) however, in these works AMR is used only implicitly, for instance, as additional features or training signals. |
| Approach: | They propose a novel AMR-based graph structure which uses graph neural networks to find event arguments from unstructured text. |
| Outcome: | The proposed graph structure outperforms the state-of-the-art models by 3.63pt and 2.33pt F1 and reduces inference time by 56%. |
An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current methods for extracting opinion words for an aspect in text leverage position embeddings to capture relative position of word to the target. |
| Approach: | They propose to use pretrained word embeddings to extract opinion words for a given aspect in text. |
| Outcome: | The proposed methods outperform current methods on a task based on pre-trained word embeddings and position embedders. |
Named Entity Recognition Only from Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing named entity recognition systems require large amounts of human annotated training data. |
| Approach: | They propose a fully unsupervised named entity recognition model which takes clues from pre-trained word embeddings. |
| Outcome: | The proposed model can be trained on two CoNLL benchmark datasets without annotating lexicon or corpus. |
CODER: An efficient framework for improving retrieval through COntextual Document Embedding Reranking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Contextual document embedding reranking is an efficient and efficient retrieval framework. |
| Approach: | They propose a highly efficient retrieval framework that uses contextual document embedding reranking to incorporate ranking context into training. |
| Outcome: | The proposed framework reduces the computational overhead of a first-stage method and can be used as stand-alone retrieval models. |
PathQG: Neural Question Generation from Facts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research for question generation encodes text as a sequence of tokens without explicitly modeling fact information. |
| Approach: | They propose to incorporate facts in the input text for question generation in a comprehensive way. |
| Outcome: | The proposed model outperforms state-of-the-art models and human evaluation shows it generates relevant and informative questions. |
Enhancing Idiomatic Representation in Multiple Languages via an Adaptive Contrastive Triplet Loss (2024.findings-acl)
Copied to clipboard
| Challenge: | Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in natural language processing (NLP). |
| Approach: | They propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning by using adaptive contrastive learning and resampling miners. |
| Outcome: | The proposed model outperforms previous models significantly on a SemEval challenge and outperformed previous alternatives in many metrics. |
Improving Quotation Attribution with Fictional Character Embeddings (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods to attribute quotes to human logic lack character representations, which often leads to errors in more challenging examples of attribution: anaphoric and implicit quotes. |
| Approach: | They propose to augment a popular quotation attribution system, BookNLP, with character embeddings that encode global stylistic information of characters derived from an off-the-shelf stylometric model, Universal Authorship Representation (UAR). |
| Outcome: | The proposed system improves anaphoric and implicit quotes, reaching state-of-the-art. |
How Retrieved Context Shapes Internal Representations in RAG (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is a widely adopted approach for enhancing large language models with external knowledge. |
| Approach: | They analyze how different types of retrieved documents affect the hidden states of large language models and how these internal representation shifts relate to downstream generation behavior. |
| Outcome: | The results show that context relevancy and layer-wise processing influence internal representations, providing explanations of LLMs’ output behaviors and insights for RAG system design. |
Should All Cross-Lingual Embeddings Speak English? (2020.acl-main)
Copied to clipboard
| Challenge: | lexicon induction evaluation dictionaries are mostly between English and another language, and the English hub is selected by default as the hub . lexiconic embeddings are often learned with a two-step process, whether under bilingual or multilingual settings. |
| Approach: | They propose to use English as the hub language for lexicon induction evaluation . they also expand a standard English-centered evaluation dictionary collection to include all language pairs . |
| Outcome: | The proposed method can significantly improve lexicon induction performance over multiple languages. |
Improving Embeddings Representations for Comparing Higher Education Curricula: A Use Case in Computing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to represent study programs using bag-of-words and clustering algorithms are prone to biases due to personal beliefs and perspectives. |
| Approach: | They propose to use pre-trained word embeddings to fine-tune a study program classification task to obtain more accurate curriculum representations than strong baselines. |
| Outcome: | The proposed method is compared to existing methods on a study program classification task and on comparing computing curricula from USA and Latin America. |
Gender Representation in Open Source Speech Resources (2020.lrec-1)
Copied to clipboard
| Challenge: | Using open source corpora, we find that gender balance depends on other corpus characteristics such as elicited/non ellicite vs. non-eliciting speech, low/high resource language, speech task targeted. |
| Approach: | They propose to use open source corpora to find gender information in spoken language systems . they propose metadata and recommendations for researchers to assure better transparency . |
| Outcome: | The proposed method improves the quality and transparency of open source speech resources. |
Italian Word Embeddings for the Medical Domain (2024.lrec-main)
Copied to clipboard
| Challenge: | Neural word embeddings have proven valuable in the development of medical applications, but for the Italian language, there are no publicly available corpora, embedds, or evaluation resources tailored to this domain. |
| Approach: | They propose to use a corpus of medical texts to generate neural word embeddings in Italian using Metathesaurus concept graphs. |
| Outcome: | The results show that the new embeddings correlate well with human judgments regarding similarity and relatedness of medical concepts. |
RepMatch: Quantifying Cross-Instance Similarities in Representation Space (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in dataset analysis have enabled more sophisticated approaches to analyzing and characterizing training data instances. |
| Approach: | They propose a method that characterizes data through the lens of similarity. |
| Outcome: | The proposed method can compare datasets, identify more representative subsets, and uncover heuristics underlying the construction of some challenge datasets. |
A New Representation for Span-based CCG Parsing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural CCG parsing methods do not support CCG derivations that violate the rule schemata. |
| Approach: | They propose a new representation for CCG derivations that decomposes CCG into several independent pieces. |
| Outcome: | The proposed representation decomposes CCG derivations into independent pieces . it prevents span-based models from violating the schemata . |
Database-Augmented Query Representation for Information Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Information retrieval models that aim to search for documents relevant to a query have shown multiple successes, but the query from the user is oftentimes short, which challenges the retrievers to correctly fetch relevant documents. |
| Approach: | They propose a database-augmented Query representation framework which augments the query with various (query-related) metadata across multiple tables. |
| Outcome: | The proposed framework significantly improves overall retrieval performance over baselines. |
Representation Potentials of Foundation Models for Multimodal Alignment: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | foundation models learn highly transferable representations through large-scale pretraining on diverse data. |
| Approach: | They examine the representation potentials of foundation models by examining their latent capacity to capture task-specific information within a single modality while providing a transferable basis for alignment and unification across modalities. |
| Outcome: | The foundation models exhibit remarkable similarities across architectures and modalities, the authors show . the models can capture task-specific information within a single modality while providing a transferable basis for alignment and unification across modality. |
A Joint Matrix Factorization Analysis of Multilingual Representations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have demonstrated that pre-trained models acquire and incorporate linguistic knowledge in their multilingual representations. |
| Approach: | They propose a tool for comparing latent representations of multilingual and monolingual models . they use joint matrix factorization to analyze multiple sets of representations in a joint manner . |
| Outcome: | The proposed tool analyzes latent representations of multilingual and monolingual models . it shows that language properties influence the factorization outputs . |
Adaptive Axes: A Pipeline for In-domain Social Stereotype Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to quantify social stereotypes have struggled to capture the variability in stereotypes across conceptual domains for the same social group. |
| Approach: | They propose to use text embedding models and adaptive semantic axes to recover stereotypes from contextual representations by using large language models. |
| Outcome: | The proposed pipeline surpasses token-based methods in capturing in-domain framing and tracks stereotypes along domain-specific semantic axes for in- domain texts. |
Change Entity-guided Heterogeneous Representation Disentangling for Change Captioning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to describe differences between two images are highly challenging due to distractors such as illumination and viewpoint changes. |
| Approach: | They propose a change-entity-guided disentanglement network that explicitly learns difference representations while mitigating the impact of distractors. |
| Outcome: | The proposed method outperforms existing methods on CLEVR-Change, CLE VR-DC and Spot-the-Diff datasets and achieves state-of-the art performance. |
Early Exit with Disentangled Representation and Equiangular Tight Frame (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing early exit paradigm relies on training parametrical internal classifiers to complete specific tasks. |
| Approach: | They propose a method to decouple two distinct types of representation and introduce a non-parametric tight frame classifier for improvement. |
| Outcome: | Experiments on monolingual and multilingual tasks show that the proposed method improves over existing methods. |
Variance Matters: Detecting Semantic Differences without Corpus/Word Alignment (2023.emnlp-main)
Copied to clipboard
| Challenge: | a new method for finding semantic differences in words appears in two corpora, but it requires a variance of word vectors . a word covers more meanings in a corpus, and its mean word vector becomes shorter . |
| Approach: | They propose a method to measure the coverage of meanings of a word in a corpus through the norm of its mean word vector. |
| Outcome: | The proposed methods rival the best-performing system in the SemEval-2020 Task 1 . they are robust for the skew in corpus sizes and capable of detecting infrequent words . |
ClimRetrieve: A Benchmarking Dataset for Information Retrieval from Corporate Climate Disclosures (2024.emnlp-main)
Copied to clipboard
| Challenge: | Qualitative disclosures typically include narrative descriptions of climate-related risks, opportunities, strategies, and governance. |
| Approach: | They simulate typical tasks of a sustainability analyst by examining 30 sustainability reports with 16 detailed climate-related questions. |
| Outcome: | The proposed model combines expert knowledge with embeddings in a dataset with over 8.5K unique question-source-answer pairs labeled by different levels of relevance. |
Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings (2025.findings-acl)
Copied to clipboard
Yubo Ma, Jinsong Li, Yuhang Zang, Xiaobao Wu, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Jiaqi Wang, Yixin Cao, Aixin Sun
| Challenge: | Visualized Document Retrieval (VDR) uses large vision-language models to encode document pages into embeddings. |
| Approach: | They evaluate methods to reduce patch embeddings per page while minimizing performance degradation. |
| Outcome: | The proposed method maintains 98.2% of retrieval performance with only 11.8% of original memory usage and preserves 94.6% effectiveness at 2% memory footprint. |
Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction (2025.findings-acl)
Copied to clipboard
| Challenge: | a novel method for encoding fine-grained error patterns improves performance on GEC. |
| Approach: | They propose a method for encoding grammatical errors from LLMs' internal states using a GER method. |
| Outcome: | The proposed method significantly boosts performance in ICL settings on multilingual GEC datasets. |
Unsupervised Discrete Representations of American Sign Language (2024.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP models use discrete tokens to represent continuous signals, such as videos, audio, or gestures . modalities that are continuous are difficult to use with discrete models, such a LLM . |
| Approach: | They propose a method that discretizes sequences of fingerspelling signs into tokens . they also propose 'loss function' to improve interpretability of the tokens. |
| Outcome: | The proposed method improves the performance of the tokenizer on downstream tasks. |
Advancing Collaborative Debates with Role Differentiation through Multi-Agent Reinforcement Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Multi-agent collaborative tasks exhibit exceptional capabilities in natural language applications and generation. |
| Approach: | They propose a multi-LLM Cooperation framework with automatic role assignment capabilities that allows multiple agents to embed roles in turn-based speaking. |
| Outcome: | The proposed framework improves collaboration and expertise among agents and teams by enabling them to share roles and develop complementary strengths from the optimization level. |
LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models generate tokens by updating high-dimensional representations and decoding from them at each timestep. |
| Approach: | They propose a framework that allows reasoning correction and length control based on derived ideal trajectories. |
| Outcome: | The proposed model can predict correctness and length control based on ideal trajectories. |
Concept Tokens: Learning Behavioral Embeddings Through Concept Definitions (2026.findings-acl)
Copied to clipboard
| Challenge: | Concept Tokens is a lightweight method that adds a special token to a pretrained LLM . we find that negating the hallucination token reduces hallucines and lowers precision . |
| Approach: | They propose a lightweight method that adds a new special token to a pretrained LLM and learns only its embedding from multiple natural language definitions of a target concept. |
| Outcome: | The proposed method can learn only its embedding from multiple definitions of a target concept . the study shows that it can improve hallucinations and recasting in closed-book questions . |
Better Embeddings with Coupled Adam (2025.acl-long)
Copied to clipboard
| Challenge: | Anisotropic Embeddings Large Language Models exhibit undesirable yet poorly understood feature of anisotropy. |
| Approach: | They propose an algorithm that uses the second moment in Adam to mitigate anisotropic embeddings . they propose an embeddable matrix and unembedding matrix to map the input and output tokens based on weight tying . |
| Outcome: | The proposed model improves quality and performance on large datasets. |
Sticking to the Mean: Detecting Sticky Tokens in Text Embedding Models (2025.acl-long)
Copied to clipboard
| Challenge: | Sticky tokens, when repeatedly inserted into sentences, pull sentence similarity toward a certain value, disrupting the normal distribution of embedding distances and degrading downstream performance. |
| Approach: | They propose a method to detect “sticky tokens” by sentence and token filtering and apply it to 40 checkpoints across 14 model families. |
| Outcome: | The proposed method detects 868 sticky tokens across 14 models and shows that their presence does not correlate with model size or vocabulary size. |
An LLM-Embedding Semantic Adaptation Network for Post-level Semantic Drift Evaluation (2026.findings-acl)
Copied to clipboard
| Challenge: | Evaluating semantic drift is essential for understanding discourse evolution and opinion formation in online discussions. |
| Approach: | They propose an LLM-embedding Semantic Adaptation Network to evaluate semantic drift . they use an LRU module, an LEM-Embedding graph convolutional network module and an adaptive fusion module to integrate features from event related posts. |
| Outcome: | The proposed model achieves state-of-the-art performance on the semantic drift evaluation task compared to baseline models. |
Unpacking Bias: An Empirical Study of Bias Measurement Metrics, Mitigation Algorithms, and Their Interactions (2024.lrec-main)
Copied to clipboard
| Challenge: | Word embeddings (WE) models reflect gender, racial, and religious stereotypes from the corpus on which they are trained. |
| Approach: | They propose a method that carefully controls for word sets and vector normalization to address these factors. |
| Outcome: | The proposed method detects consistency between different mitigation methods and the evaluation words used by the mitigation methods. |
Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings (2025.acl-long)
Copied to clipboard
| Challenge: | Text embedding models are used for various natural language processing tasks such as sentiment analysis, text clustering, and content-based information retrieval. |
| Approach: | They propose a synthesis framework that leverages large language models to generate diverse negative samples with varying levels of similarity with the query. |
| Outcome: | The proposed framework achieves state-of-the-art performance surpassing existing synthesis strategies with synthetic data and when combined with public retrieval datasets. |
Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to cross-modal image-text retrieval struggle with nuanced cross-modal relationships. |
| Approach: | They propose a set-based approach that represents each sample with multiple embeddings to capture nuanced and diverse relationships. |
| Outcome: | The proposed method achieves state-of-the-art performance on MS-COCO and Flickr30k without external data. |
X-CoT: Explainable Text-to-Video Retrieval via LLM-based Chain-of-Thought Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-video retrieval systems use embedding models for feature extraction and compute cosine similarities for ranking. |
| Approach: | They propose an explainable retrieval framework upon LLM CoT reasoning to replace embedding models for feature extraction and ranking. |
| Outcome: | The proposed retrieval framework improves retrieval performance and produces detailed rationales. |
Cross-lingual Matryoshka Representation Learning across Speech and Text (2026.findings-acl)
Copied to clipboard
| Challenge: | Speakers of under-represented languages face language barriers and modality barriers . we train a bilingual speech-text embedding model for French-Wolof . |
| Approach: | They train a bilingual speech-text Matryoshka embedding model that enables efficient retrieval of French text from Wolof speech queries. |
| Outcome: | The proposed model can retrieve French text from Wolof speech queries without expensive ASR-translation pipelines. |
RATE: Reviewer Profiling and Annotation-free Training for Expertise Ranking in Peer Review Systems (2026.acl-long)
Copied to clipboard
| Challenge: | LR-bench is a high-fidelity, up-to-date benchmark curated from 2024–2025 AI/NLP manuscripts with five-level self-assessed familiarity ratings collected via a large-scale email survey . |
| Approach: | They propose a reviewer-centric ranking framework that distills each reviewer’s recent publications into compact keyword-based profiles and fine-tunes an embedding model with weak preference supervision constructed from heuristic retrieval signals. |
| Outcome: | The proposed framework outperforms existing benchmarks and the CMU gold-standard dataset in the evaluation of AI/NLP manuscripts. |
Words that make SENSE: Sensorimotor Norms in Learned Lexical Token Representations (2026.findings-acl)
Copied to clipboard
| Challenge: | Empirical studies suggest that comprehending action, perceptual and abstract concepts elicits rapid, automatic activity in modality-specific brain areas. |
| Approach: | They propose a model that predicts Lancaster sensorimotor norms from word lexical embeddings. |
| Outcome: | The proposed model predicts Lancaster sensorimotor norms from word lexical embeddings. |
SafeConstellations: Mitigating Over-Refusals in LLMs Through Task-Aware Representation Steering (2026.acl-long)
Copied to clipboard
| Challenge: | Current safety alignment methods fail to identify intended benign task before refusing to respond. |
| Approach: | They propose a method that uses inference-time trajectory-shifting to guide model behavior . they show that LLMs persist in refusing inputs containing harmful content . |
| Outcome: | The proposed approach reduces over-refusals with minimal impact on utility. |
From Representation to Choice: Tracing Decision Emergence Across Languages in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have made them highly multilingual, but how they internally reason remains unexplored. |
| Approach: | They propose to model multilingual reasoning through a decision-making perspective using aligned multiple-choice questions from the mMMLU benchmark. |
| Outcome: | The proposed model shows that languages share similar activation spaces, but subtle divergences emerge as decisions propagate through transformer layers. |
Uncovering Currency Bias and Syntax Gap in Text Embedding Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Text-embedding models often inherit societal biases, yet the influence of socio-economic markers remains unexplored. |
| Approach: | They propose to identify Currency Bias as a systemic representational limitation in financial AI . they analyze currency embeddings to identify currency identifiers and associative sensitivity . |
| Outcome: | The proposed model lacks associative sensitivity to economic hierarchies, the authors show . they show that current embedding practices pose significant risks for the fairness and reliability of financial NLP applications. |
Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multimodal affective computing learn spurious correlations from training data rather than genuine causal relationships, harming generalization under distribution shifts or noisy modalities. |
| Approach: | They propose a causal modality-invariant representation framework that separates each modality into ‘causal invariant’ and ‘environment-specific spurious representation’ from a modal inference perspective. |
| Outcome: | Experiments on multiple multimodal benchmarks show that the proposed framework achieves state-of-the-art performance. |